Nov 02 21:31:35.716712 ip-10-0-136-74 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 02 21:31:35.716721 ip-10-0-136-74 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 02 21:31:35.716728 ip-10-0-136-74 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 02 21:31:35.716935 ip-10-0-136-74 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 02 21:31:45.794862 ip-10-0-136-74 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 02 21:31:45.794876 ip-10-0-136-74 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7cabbb072d5e404ba262bca470193391 --
Nov 02 21:33:48.823213 ip-10-0-136-74 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 21:33:49.208242 ip-10-0-136-74 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:33:49.208242 ip-10-0-136-74 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 21:33:49.208242 ip-10-0-136-74 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:33:49.208242 ip-10-0-136-74 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:33:49.211476 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211398    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.211476 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211475    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.211476 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211479    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211482    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211485    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211488    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211491    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211495    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211499    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211502    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211505    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211508    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211510    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211513    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211516    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211518    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211521    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211523    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211525    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211528    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211530    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211533    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.211566 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211535    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211538    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211541    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211543    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211553    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211556    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211558    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211561    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211563    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211566    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211568    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211571    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211574    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211577    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211580    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211582    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211585    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211588    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211590    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.212047 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211593    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211595    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211598    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211600    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211603    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211605    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211607    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211610    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211612    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211614    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211617    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211619    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211622    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211624    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211627    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211630    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211633    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211636    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211638    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.212485 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211642    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211644    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211647    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211649    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211652    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211654    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211657    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211659    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211661    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211665    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211668    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211673    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211676    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211678    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211681    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211683    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211686    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211688    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211691    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211693    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.212918 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211696    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211698    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211701    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211703    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211706    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211708    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211711    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211714    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211717    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211720    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211722    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211725    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211728    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211730    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211733    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211735    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211738    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211740    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211743    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.213393 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211745    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211747    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211750    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211753    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211755    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211758    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.211760    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212504    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212510    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212513    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212516    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212518    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212521    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212523    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212526    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212529    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212533    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212535    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212538    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.213855 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212541    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212544    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212547    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212549    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212552    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212554    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212557    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212570    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212574    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212578    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212581    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212584    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212586    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212589    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212591    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212594    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212596    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212599    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212601    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.214356 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212604    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212606    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212609    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212611    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212614    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212616    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212619    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212621    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212624    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212630    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212633    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212636    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212639    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212641    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212643    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212646    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212649    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212651    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212654    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212656    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.214803 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212658    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212661    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212663    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212666    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212668    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212671    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212674    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212676    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212679    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212681    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212684    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212686    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212688    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212691    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212693    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212695    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212698    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212700    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212703    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212705    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.215272 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212707    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212711    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212721    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212725    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212728    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212731    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212733    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212736    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212738    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212741    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212743    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212746    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212749    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212751    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212754    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212756    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212758    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212761    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212763    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212766    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.215733 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212768    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212770    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212773    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212775    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212778    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212780    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212782    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212785    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212787    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212790    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212792    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212795    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212799    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212803    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.212806    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213300    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213306    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213309    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213312    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.216210 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213315    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213318    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213320    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213323    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213326    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213328    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213331    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213333    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213336    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213339    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213341    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213344    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213346    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213348    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213351    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213353    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213356    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213358    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213361    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.216640 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213363    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213365    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213368    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213370    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213373    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213375    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213378    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213381    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213383    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213387    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213390    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213392    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213395    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213398    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213400    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213403    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213405    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213408    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213411    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213413    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.217112 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213416    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213418    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213421    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213423    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213425    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213428    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213430    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213432    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213435    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213437    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213440    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213442    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213445    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213447    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213449    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213452    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213454    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213456    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213459    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.217571 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213462    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213464    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213467    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213473    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213476    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213478    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213481    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213483    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213486    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213488    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213491    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213494    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213496    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213499    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213502    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213504    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213507    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213509    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213511    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213514    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.218018 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213516    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213519    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213521    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213523    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213527    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213530    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213533    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213536    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213538    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213541    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213543    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213546    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213549    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213551    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213555    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213559    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213562    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213564    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213567    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.218525 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213569    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213572    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213578    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213581    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.213583    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214369    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214379    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214389    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214394    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214399    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214402    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214406    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214410    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214413    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214417    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214420    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214423    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214426    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214429    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214432    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214435    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214438    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214441    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 21:33:49.218982 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214448    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214451    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214455    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214458    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214461    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214465    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214468    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214472    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214475    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214478    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214481    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214486    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214489    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214492    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214496    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214499    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214502    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214505    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214508    2580 flags.go:64] FLAG: --enable-server="true"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214511    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214516    2580 flags.go:64] FLAG: --event-burst="100"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214519    2580 flags.go:64] FLAG: --event-qps="50"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214522    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214525    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 21:33:49.219493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214528    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214532    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214535    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214538    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214541    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214544    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214548    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214551    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214554    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214557    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214560    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214563    2580 flags.go:64] FLAG: --feature-gates=""
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214567    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214570    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214573    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214576    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214580    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214583    2580 flags.go:64] FLAG: --help="false"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214586    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214590    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214593    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214596    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214600    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 21:33:49.220079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214603    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214606    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214609    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214612    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214615    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214618    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214620    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214623    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214626    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214629    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214632    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214634    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214637    2580 flags.go:64] FLAG: --lock-file=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214640    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214643    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214647    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214652    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214655    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214658    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214661    2580 flags.go:64] FLAG: --logging-format="text"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214664    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214667    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214670    2580 flags.go:64] FLAG: --manifest-url=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214673    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214677    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 21:33:49.220648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214680    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214684    2580 flags.go:64] FLAG: --max-pods="110"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214687    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214695    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214699    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214701    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214704    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214707    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214710    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214717    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214720    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214723    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214726    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214729    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214732    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214735    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214738    2580 flags.go:64] FLAG: --port="10250"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214741    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214744    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-010ad93c5847a9a8e"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214747    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214750    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214753    2580 flags.go:64] FLAG: --register-node="true"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214757    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214761    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214764    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214766    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 21:33:49.221259 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214769    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214773    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214776    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214779    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214782    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214785    2580 flags.go:64] FLAG: --runonce="false"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214788    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214791    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214795    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214799    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214802    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214806    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214809    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214813    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214816    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214819    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214821    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214824    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214827    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214830    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214833    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214838    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214841    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214843    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214848    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 02 21:33:49.221889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214850    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214854    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214857    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214860    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214863    2580 flags.go:64] FLAG: --v="2"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214867    2580 flags.go:64] FLAG: --version="false"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214871    2580 flags.go:64] FLAG: --vmodule=""
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214875    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.214878    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215023    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215028    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215031    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215034    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215036    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215039    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215041    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215046    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215048    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215051    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215054    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215056    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.222478 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215058    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215061    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215063    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215066    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215068    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215071    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215073    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215077    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215081    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215084    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215087    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215089    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215093    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215097    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215100    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215103    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215106    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215109    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215111    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.222981 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215114    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215116    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215119    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215121    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215124    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215126    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215129    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215132    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215135    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215138    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215141    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215143    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215146    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215148    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215150    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215153    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215155    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215157    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215160    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.223429 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215162    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215164    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215167    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215169    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215172    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215175    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215177    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215180    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215182    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215185    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215187    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215190    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215192    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215194    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215197    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215200    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215202    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215205    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215207    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.223871 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215209    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215212    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215215    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215218    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215221    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215223    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215225    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215228    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215230    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215233    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215235    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215238    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215240    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215242    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215245    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215247    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215250    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215252    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215255    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215258    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.224324 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215260    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215263    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215265    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215268    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215270    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215273    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215275    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215278    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215280    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215283    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215286    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215288    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215291    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215293    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215296    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215299    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.224782 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215301    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.215327    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215361    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215365    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215368    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215371    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215373    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215376    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215378    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215381    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215383    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215386    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215388    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215391    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215395    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215398    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.225171 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215400    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215403    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215405    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215408    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215410    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215413    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215415    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215418    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215420    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215423    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215425    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215428    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215430    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215433    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215435    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215438    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215440    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215442    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215445    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.225537 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215448    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215451    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215453    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215456    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215458    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215460    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215463    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215465    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215468    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215470    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215473    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215477    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215479    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215482    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215484    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215487    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215489    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215492    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215494    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215497    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.225988 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215499    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215502    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215504    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215507    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215509    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215511    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215514    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215517    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215520    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215522    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215525    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215527    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215530    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215532    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215535    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215537    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215540    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215542    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215545    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215547    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.226527 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215550    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215552    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215555    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215559    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215561    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215564    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215567    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215571    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215574    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215576    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215579    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215583    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215587    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215590    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215592    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215595    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215597    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215600    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215602    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.227010 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215605    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215607    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215610    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215612    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215615    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215618    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215621    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215623    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215626    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215628    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215631    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215634    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215637    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.215639    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.221870    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.221885    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.221900    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.221905    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221975    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.227488 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221980    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221984    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221987    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221990    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221993    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221995    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.221998    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222000    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222003    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222006    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222008    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222011    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222013    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222016    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222018    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222021    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222024    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222026    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222028    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222031    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.227920 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222034    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222037    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222040    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222042    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222046    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222048    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222051    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222054    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222057    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222060    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222062    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222065    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222067    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222070    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222072    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222075    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222077    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222080    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222082    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.228397 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222085    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222089    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222092    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222095    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222097    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222100    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222102    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222106    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222110    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222113    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222116    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222118    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222121    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222123    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222126    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222129    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222131    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222134    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222136    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.228836 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222139    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222142    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222145    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222147    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222150    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222152    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222155    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222157    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222160    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222162    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222165    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222167    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222170    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222172    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222175    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222177    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222180    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222182    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222184    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222187    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.229530 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222189    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222192    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222194    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222196    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222199    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222201    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222204    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222206    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222209    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222212    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222214    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222216    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222219    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222222    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222225    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222228    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222230    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222233    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222235    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.230307 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222238    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222241    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222244    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222246    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222249    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222251    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222254    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222256    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.222282    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222418    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222423    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222427    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222430    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222432    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222435    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.230858 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222438    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222440    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222442    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222445    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222447    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222450    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222468    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222472    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222475    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222478    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222481    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222484    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222487    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222490    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222493    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222495    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222499    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222504    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222507    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222510    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.231319 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222512    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222515    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222517    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222520    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222522    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222525    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222527    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222530    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222532    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222535    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222537    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222540    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222542    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222544    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222547    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222549    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222551    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222554    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222556    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222559    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.231822 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222561    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222564    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222566    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222569    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222572    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222575    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222577    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222580    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222582    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222585    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222588    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222590    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222592    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222595    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222597    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222599    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222602    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222604    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222607    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.232562 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222609    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222611    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222614    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222616    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222619    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222621    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222623    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222626    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222628    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222631    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222633    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222636    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222638    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222640    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222643    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222646    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222648    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222651    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222653    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222655    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.233039 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222658    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222660    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222663    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222665    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222667    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222670    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222672    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222675    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222677    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222679    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222682    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222684    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222686    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222689    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222691    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222694    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222696    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222698    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222701    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222703    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.233501 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222706    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.222728    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222757    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222761    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222764    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222766    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222769    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222772    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222774    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222777    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222780    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222782    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222785    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222787    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222790    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222793    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:33:49.234510 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222795    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222797    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222800    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222802    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222804    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222807    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222809    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222811    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222814    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222816    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222819    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222821    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222823    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222826    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222828    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222831    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222834    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222836    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222838    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:33:49.234892 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222841    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222843    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222846    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222848    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222851    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222855    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222857    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222860    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222862    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222865    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222867    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222869    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222872    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222874    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222877    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222879    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222881    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222884    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222886    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222889    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:33:49.235479 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222891    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222893    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222895    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222898    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222900    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222903    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222905    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222908    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222910    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222915    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222918    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222920    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222922    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222925    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222927    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222930    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222933    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222935    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222938    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222940    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:33:49.236002 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222943    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222945    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222948    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222950    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222953    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222971    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222976    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222979    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222982    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222985    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222988    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222991    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222993    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222996    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.222999    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223001    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223004    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223007    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223009    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:33:49.236601 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223011    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223014    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223017    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223020    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223023    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223025    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223028    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223030    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223035    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223038    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223041    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223044    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223047    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:49.223050    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.223099    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.226191    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.227033    2580 server.go:1014] "Starting client certificate rotation"
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.227125    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:33:49.237121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.227893    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:33:49.237559 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.229618    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:33:49.237559 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.232454    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:33:49.242889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.242872    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 02 21:33:49.246409 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.246397    2580 log.go:25] "Validated CRI v1 image API"
Nov 02 21:33:49.247147 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.247122    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 21:33:49.252034 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.252015    2580 fs.go:135] Filesystem UUIDs: map[1485b32b-8ea0-4aff-96fc-2e645beb302a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b45cc05b-d07a-41ff-80d1-7c53c2bec687:/dev/nvme0n1p3]
Nov 02 21:33:49.252084 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.252034    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 21:33:49.256939 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.256828    2580 manager.go:217] Machine: {Timestamp:2026-11-02 21:33:49.255571998 +0000 UTC m=+0.315768622 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3197173 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd87621b33c7e81b7e0b3dbf03915 SystemUUID:ec2cd876-21b3-3c7e-81b7-e0b3dbf03915 BootID:7cabbb07-2d5e-404b-a262-bca470193391 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:ac:39:22:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:ac:39:22:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:46:94:ec:71:4c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 21:33:49.257020 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.256941    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 21:33:49.257067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.257055    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 21:33:49.260066 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260039    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 21:33:49.260198 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260067    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-74.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 21:33:49.260243 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260201    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 21:33:49.260243 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260208    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 21:33:49.260297 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260243    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:33:49.260297 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.260261    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:33:49.261060 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.261049    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 21:33:49.264986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.264974    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 21:33:49.265096 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265087    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 21:33:49.265745 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265736    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 21:33:49.265811 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265749    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 21:33:49.265811 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265760    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 21:33:49.265811 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265769    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 02 21:33:49.265811 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.265778    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 21:33:49.266140 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.266126    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:33:49.268213 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.268199    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 21:33:49.273193 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.273173    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:33:49.274056 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274040    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 21:33:49.274104 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274074    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 21:33:49.274485 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274471    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 21:33:49.274519 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274492    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 21:33:49.274519 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274499    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 21:33:49.274519 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274506    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 21:33:49.274519 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274516    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 21:33:49.274619 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274526    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 21:33:49.274990 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274980    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 21:33:49.275020 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274991    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 21:33:49.275020 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.274999    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 21:33:49.275020 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.275005    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 21:33:49.275020 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.275020    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 21:33:49.275142 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.275029    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 21:33:49.275571 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.275562    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 21:33:49.275601 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.275572    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 21:33:49.279321 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.279309    2580 server.go:1263] "Started kubelet"
Nov 02 21:33:49.279395 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.279367    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 21:33:49.279449 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.279400    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 21:33:49.279489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.279477    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 21:33:49.280046 ip-10-0-136-74 systemd[1]: Started Kubernetes Kubelet.
Nov 02 21:33:49.280683 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.280666    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 21:33:49.282045 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.282033    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 21:33:49.286669 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.286224    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 21:33:49.287019 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.286794    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 21:33:49.287282 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.287265    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 21:33:49.288350 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.288331    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 21:33:49.288350 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.288343    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:49.288498 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.288350    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 21:33:49.288498 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.288367    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 21:33:49.288600 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.288580    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 21:33:49.288600 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.288596    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 21:33:49.289440 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289425    2580 factory.go:55] Registering systemd factory
Nov 02 21:33:49.289440 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289442    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 02 21:33:49.289644 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289632    2580 factory.go:153] Registering CRI-O factory
Nov 02 21:33:49.289644 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289645    2580 factory.go:223] Registration of the crio container factory successfully
Nov 02 21:33:49.289761 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289698    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 21:33:49.289761 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289712    2580 factory.go:103] Registering Raw factory
Nov 02 21:33:49.289761 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.289722    2580 manager.go:1199] Started watching for new ooms in manager
Nov 02 21:33:49.290643 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.290629    2580 manager.go:319] Starting recovery of all containers
Nov 02 21:33:49.301541 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.301516    2580 manager.go:324] Recovery completed
Nov 02 21:33:49.307863 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.307848    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.310206 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310189    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.310266 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310224    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.310266 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310237    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.310702 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310691    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 02 21:33:49.310702 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310700    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 21:33:49.310779 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310716    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 21:33:49.310779 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.310758    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.311466 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.311451    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.311515 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.311481    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.311515 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.311494    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.313004 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.312991    2580 policy_none.go:50] "Start"
Nov 02 21:33:49.313051 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.313007    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 21:33:49.313051 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.313017    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 21:33:49.315003 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.314967    2580 policy_none.go:44] "Start"
Nov 02 21:33:49.348666 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.348632    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 21:33:49.350929 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.350914    2580 manager.go:341] "Starting Device Plugin manager"
Nov 02 21:33:49.351021 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.350948    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 21:33:49.351021 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.350974    2580 server.go:87] "Starting device plugin registration server"
Nov 02 21:33:49.351247 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.351233    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 21:33:49.351317 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.351249    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 21:33:49.351371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.351357    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 21:33:49.351464 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.351448    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 21:33:49.351464 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.351461    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 21:33:49.352137 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.352105    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 21:33:49.352202 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.352170    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:49.399760 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.399735    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 21:33:49.400890 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.400871    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 21:33:49.400949 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.400898    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 21:33:49.400949 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.400919    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 21:33:49.401050 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.400971    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 21:33:49.451703 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.451677    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.452510 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.452495    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.452578 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.452524    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.452578 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.452534    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.452578 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.452559    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.469745 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.469655    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-74.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 21:33:49.469745 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.469706    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-74.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 21:33:49.470372 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.469582    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-74.us-west-1.compute.internal.18e457ed86b6a764  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-74.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-74.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 21:33:49.279283044 +0000 UTC m=+0.339479668,LastTimestamp:2026-11-02 21:33:49.279283044 +0000 UTC m=+0.339479668,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-74.us-west-1.compute.internal,}"
Nov 02 21:33:49.502110 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.502072    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"]
Nov 02 21:33:49.502225 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.502140    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.504921 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.504904    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.504998 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.504936    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.504998 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.504951    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.506635 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.506624    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.506765 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.506751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.506815 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.506780    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.507403 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507385    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.507488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507413    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.507488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507423    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.507488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507391    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.507488 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507482    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.507619 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.507493    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.509329 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.509314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.509409 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.509340    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.510070 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.510055    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.510143 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.510083    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.510143 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.510096    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.537033 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.537015    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.542675 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.542657    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.549050 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.549035    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 21:33:49.549126 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.549074    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pk6fh"
Nov 02 21:33:49.574200 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.574127    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-74.us-west-1.compute.internal.18e457ed888e8157  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-74.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-74.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-74.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 21:33:49.310206295 +0000 UTC m=+0.370402922,LastTimestamp:2026-11-02 21:33:49.310206295 +0000 UTC m=+0.370402922,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-74.us-west-1.compute.internal,}"
Nov 02 21:33:49.574786 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.574757    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-74.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 02 21:33:49.574786 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.574757    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 02 21:33:49.574786 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.574763    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 02 21:33:49.590044 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.590012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.590132 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.590053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.590132 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.590077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.627163 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.627139    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:33:49.627708 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.627692    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-136-74.us-west-1.compute.internal\" not found" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691070 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691145    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.691171 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.691156    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.774546 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.774487    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-74.us-west-1.compute.internal\" not found" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.828633 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.828602    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:49.829611 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.829595    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:49.829696 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.829625    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:49.829696 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.829635    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:49.829696 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.829659    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.839859 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.839839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.847528 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.847512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.886772 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:49.886753    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:49.886855 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:49.886777    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-74.us-west-1.compute.internal\": node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.011055 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.010868    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.111274 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.111191    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.212148 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.212111    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.227280 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.227253    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 21:33:50.227380 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.227364    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 21:33:50.227523 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.227443    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a844a01bd9c1a45928f36aeb9963b228-b8a9b0af8073bf35.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-136-74.us-west-1.compute.internal.18e457ed888e8157\": read tcp 10.0.136.74:55742->3.14.232.160:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-136-74.us-west-1.compute.internal.18e457ed888e8157  default   5324 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-74.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-74.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-74.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 21:33:49 +0000 UTC,LastTimestamp:2026-11-02 21:33:49.504920664 +0000 UTC m=+0.565117288,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-74.us-west-1.compute.internal,}"
Nov 02 21:33:50.313114 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.313091    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.413707 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.413632    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.514668 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.514637    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.553489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.553429    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:28:49 +0000 UTC" deadline="2028-08-16 10:04:11.989332694 +0000 UTC"
Nov 02 21:33:50.553489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.553485    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15660h30m21.435852269s"
Nov 02 21:33:50.555578 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.555562    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:33:50.556577 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.556557    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:33:50.556670 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.556589    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:33:50.556670 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.556605    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:33:50.615363 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.615331    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.640816 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.640798    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:33:50.715482 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.715414    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.816475 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.816446    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:50.872586 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:50.872548    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed0968c21495795a0292b52f44b9b55.slice/crio-a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2 WatchSource:0}: Error finding container a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2: Status 404 returned error can't find the container with id a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2
Nov 02 21:33:50.872818 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:50.872800    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4a2511be029007da6e29235975f6dc.slice/crio-742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43 WatchSource:0}: Error finding container 742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43: Status 404 returned error can't find the container with id 742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43
Nov 02 21:33:50.878907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:50.878890    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:33:50.917351 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:50.917321    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:51.017998 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.017907    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:33:51.033121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.033102    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:33:51.088855 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.088827    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:51.126541 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.126518    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:33:51.148340 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.148317    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:33:51.149481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.149466    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:33:51.209325 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.209303    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:33:51.266624 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.266599    2580 apiserver.go:52] "Watching apiserver"
Nov 02 21:33:51.287772 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.287713    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 21:33:51.370511 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.370486    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 21:33:51.370806 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.370789    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-7t4gd","openshift-cluster-node-tuning-operator/tuned-pbvpg","openshift-multus/multus-additional-cni-plugins-dcf5t","openshift-multus/multus-hkp7r","kube-system/global-pull-secret-syncer-ljkpr","kube-system/konnectivity-agent-wc9ll","openshift-multus/network-metrics-daemon-x2887","openshift-network-operator/iptables-alerter-vkrw9","openshift-ovn-kubernetes/ovnkube-node-pcn6c","kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc","openshift-dns/node-resolver-rwxvq","openshift-image-registry/node-ca-c65l7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"]
Nov 02 21:33:51.374520 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.374501    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.374625 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.374594    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:33:51.376667 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.376650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.378856 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.378834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.381190 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.381175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.383795 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.383775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.383883 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.383841    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:33:51.386139 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.386116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.390520 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.390500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.393242 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.393221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:51.393313 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.393294    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:33:51.395418 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.395400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.397505 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.397489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.399661 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.399746 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.399746 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.399746 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.399746 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.399950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597wm\" (UniqueName: \"kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.399950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.399950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.399950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.399950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.399999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.400338 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400592 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9gfl\" (UniqueName: \"kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.400986 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.400918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkzxs\" (UniqueName: \"kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbkqw\" (UniqueName: \"kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl9zx\" (UniqueName: \"kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.401503 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.401428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.402213 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.402199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.406424 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.406385    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerStarted","Data":"742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43"}
Nov 02 21:33:51.407783 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.407758    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"8ed0968c21495795a0292b52f44b9b55","Type":"ContainerStarted","Data":"a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2"}
Nov 02 21:33:51.429623 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.429596    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 21:33:51.429623 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.429596    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.429734 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.429607    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.429734 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.429644    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sb4td\""
Nov 02 21:33:51.431328 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431314    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.431426 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431355    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.431489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431475    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 21:33:51.432786 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431783    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 21:33:51.432786 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431878    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 21:33:51.432786 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.431942    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qbt7b\""
Nov 02 21:33:51.433516 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.433492    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-7clxp\""
Nov 02 21:33:51.433608 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.433563    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 21:33:51.438532 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.438517    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 21:33:51.438618 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.438546    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-b8lsz\""
Nov 02 21:33:51.438618 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.438564    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 21:33:51.442894 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.442881    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 21:33:51.443341 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443328    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 21:33:51.443435 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443422    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.443483 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443471    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.443522 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443490    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 21:33:51.443522 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443517    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dsxz6\""
Nov 02 21:33:51.443751 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.443739    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 21:33:51.447847 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.447833    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.447898 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.447849    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-95jdp\""
Nov 02 21:33:51.447938 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.447915    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.449650 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.449636    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.449788 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.449775    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.449904 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.449893    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 21:33:51.450009 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.449997    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nr5z4\""
Nov 02 21:33:51.451849 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.451838    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.452049 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.452036    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 21:33:51.452134 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.452121    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.455190 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.455178    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f7c29\""
Nov 02 21:33:51.455261 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.455252    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 21:33:51.489502 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.489482    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 21:33:51.502152 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.502274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.502274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597wm\" (UniqueName: \"kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.502274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502330    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502330    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502385    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502481 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502470    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns6bz\" (UniqueName: \"kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502650    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502735    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502763    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502806    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.502937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502868    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502895    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502914    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502970    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.502988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503165    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.503641 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503292    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzxs\" (UniqueName: \"kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.503343    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.503369    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.503423    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:52.003385663 +0000 UTC m=+3.063582301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503491    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.503512    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:33:52.003471232 +0000 UTC m=+3.063667858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkqw\" (UniqueName: \"kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl9zx\" (UniqueName: \"kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kr9g\" (UniqueName: \"kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503644    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.504363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhlwx\" (UniqueName: \"kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503905    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.503999    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504033    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504039    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.504907 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504126    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504162    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504280    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504328    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504362    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504453    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504480    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504526    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9gfl\" (UniqueName: \"kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504667    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504812    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504865    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.504993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505003    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.505988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505195    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505263    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505260    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdsrg\" (UniqueName: \"kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505311    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.506470 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.505376    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.507185 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.507168    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.507248 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.507228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.524240 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.524217    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 21:33:51.524326 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.524277    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-49dcf"
Nov 02 21:33:51.530201 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.530185    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 21:33:51.530278 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.530265    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2mch2\""
Nov 02 21:33:51.559616 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.559545    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597wm\" (UniqueName: \"kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.560063 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.560042    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzxs\" (UniqueName: \"kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:51.560193 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.560176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkqw\" (UniqueName: \"kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.560270 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.560251    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl9zx\" (UniqueName: \"kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.561166 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.561150    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9gfl\" (UniqueName: \"kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.605842 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.605817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.605842 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.605845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6bz\" (UniqueName: \"kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.606035 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.605863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.606035 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.605878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:51.606035 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.605894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606035 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606057    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606109    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606138    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kr9g\" (UniqueName: \"kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606204    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606227    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.606249 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlwx\" (UniqueName: \"kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606343    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606373    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606394    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606437    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606469    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606593    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.606865 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606657    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606669    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdsrg\" (UniqueName: \"kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606719    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606772    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606816    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606887    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.606973    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.607371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.607053    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.608004 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.607985    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.608278 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.608264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.662285 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.662254    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:33:51.662285 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.662284    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:33:51.662553 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.662297    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:51.662553 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:51.662371    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:52.162351816 +0000 UTC m=+3.222548460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:51.663599 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.663570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6bz\" (UniqueName: \"kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.663674 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.663598    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kr9g\" (UniqueName: \"kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.663762 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.663738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdsrg\" (UniqueName: \"kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.664115 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.664101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlwx\" (UniqueName: \"kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.687034 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.687011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:33:51.696481 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.696452    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4905999a_0718_4c3d_8cfd_abe292d5e01c.slice/crio-88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5 WatchSource:0}: Error finding container 88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5: Status 404 returned error can't find the container with id 88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5
Nov 02 21:33:51.705705 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.705684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hkp7r"
Nov 02 21:33:51.708621 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.708520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:51.711452 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.711435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:33:51.714769 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.714747    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942b2063_cdbd_49a9_bb85_21053e8f15c6.slice/crio-ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f WatchSource:0}: Error finding container ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f: Status 404 returned error can't find the container with id ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f
Nov 02 21:33:51.720433 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.720411    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab35d4ec_3432_43b7_adca_34097c507ae1.slice/crio-080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7 WatchSource:0}: Error finding container 080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7: Status 404 returned error can't find the container with id 080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7
Nov 02 21:33:51.720686 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.720606    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88de0d5_7833_4038_8119_201c5fff01f5.slice/crio-a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818 WatchSource:0}: Error finding container a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818: Status 404 returned error can't find the container with id a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818
Nov 02 21:33:51.722175 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.722070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:33:51.723673 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.723628    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:33:51.732013 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.731992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:33:51.732615 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.732593    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103a29ef_eca9_4925_93df_023ee970d7f7.slice/crio-c7515618e360daebde221f62bf7affb8c117a38bbd098c3a6b3beb6ade8b1be3 WatchSource:0}: Error finding container c7515618e360daebde221f62bf7affb8c117a38bbd098c3a6b3beb6ade8b1be3: Status 404 returned error can't find the container with id c7515618e360daebde221f62bf7affb8c117a38bbd098c3a6b3beb6ade8b1be3
Nov 02 21:33:51.733449 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.733428    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35ac1d5_682e_4eab_9a5b_a0675b542860.slice/crio-5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8 WatchSource:0}: Error finding container 5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8: Status 404 returned error can't find the container with id 5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8
Nov 02 21:33:51.738837 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.738822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:33:51.740638 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.740619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:33:51.740866 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.740846    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc7319d_94df_4f5d_88bb_9b85e9ee0eb7.slice/crio-e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9 WatchSource:0}: Error finding container e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9: Status 404 returned error can't find the container with id e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9
Nov 02 21:33:51.750496 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.750474    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a0fc7f_ccb0_4fd0_bcce_5a13f861c83c.slice/crio-3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f WatchSource:0}: Error finding container 3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f: Status 404 returned error can't find the container with id 3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f
Nov 02 21:33:51.750732 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:33:51.750712    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef94b9d_be65_4a3d_a03a_2ad5ffd5a7e9.slice/crio-8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e WatchSource:0}: Error finding container 8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e: Status 404 returned error can't find the container with id 8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e
Nov 02 21:33:51.851863 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:51.851783    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:33:52.010070 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.010031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:52.010070 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.010079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:52.010312 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.010203    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:52.010312 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.010219    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:52.010312 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.010276    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:33:53.010255815 +0000 UTC m=+4.070452445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:52.010312 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.010300    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:53.010288848 +0000 UTC m=+4.070485471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:52.211946 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.211860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:52.212131 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.212039    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:33:52.212131 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.212054    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:33:52.212131 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.212063    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:52.212131 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.212109    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:53.21209623 +0000 UTC m=+4.272292855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:52.401716 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.401686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:52.402200 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:52.401821    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:33:52.410654 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.410613    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rwxvq" event={"ID":"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c","Type":"ContainerStarted","Data":"3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f"}
Nov 02 21:33:52.412088 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.411984    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9"}
Nov 02 21:33:52.413377 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.413351    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vkrw9" event={"ID":"b35ac1d5-682e-4eab-9a5b-a0675b542860","Type":"ContainerStarted","Data":"5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8"}
Nov 02 21:33:52.414760 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.414712    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerStarted","Data":"a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818"}
Nov 02 21:33:52.416013 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.415992    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wc9ll" event={"ID":"ab35d4ec-3432-43b7-adca-34097c507ae1","Type":"ContainerStarted","Data":"080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7"}
Nov 02 21:33:52.417467 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.417442    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hkp7r" event={"ID":"942b2063-cdbd-49a9-bb85-21053e8f15c6","Type":"ContainerStarted","Data":"ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f"}
Nov 02 21:33:52.418981 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.418947    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc" event={"ID":"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9","Type":"ContainerStarted","Data":"8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e"}
Nov 02 21:33:52.420440 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.420387    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pbvpg" event={"ID":"103a29ef-eca9-4925-93df-023ee970d7f7","Type":"ContainerStarted","Data":"c7515618e360daebde221f62bf7affb8c117a38bbd098c3a6b3beb6ade8b1be3"}
Nov 02 21:33:52.421648 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.421616    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c65l7" event={"ID":"4905999a-0718-4c3d-8cfd-abe292d5e01c","Type":"ContainerStarted","Data":"88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5"}
Nov 02 21:33:52.529113 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.529002    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:28:51 +0000 UTC" deadline="2028-07-29 05:16:44.759357874 +0000 UTC"
Nov 02 21:33:52.529113 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:52.529059    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15223h42m52.23030246s"
Nov 02 21:33:53.018889 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.018851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:53.019068 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.018900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:53.019129 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.019089    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:53.019174 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.019147    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:55.019129867 +0000 UTC m=+6.079326481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:53.019230 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.019203    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:53.019277 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.019234    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:33:55.019225051 +0000 UTC m=+6.079421667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:53.220130 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.220050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:53.220332 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.220211    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:33:53.220332 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.220226    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:33:53.220332 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.220240    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:53.220332 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.220300    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:55.22028223 +0000 UTC m=+6.280478847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:53.404688 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.404341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:53.404688 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.404477    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:33:53.404688 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.404554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:53.404688 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:53.404618    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:33:53.529270 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.529170    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:28:51 +0000 UTC" deadline="2028-06-19 15:54:48.313406275 +0000 UTC"
Nov 02 21:33:53.529270 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:53.529224    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14274h20m54.784185932s"
Nov 02 21:33:54.402381 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:54.402119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:54.402548 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:54.402513    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:33:54.434900 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:54.434822    2580 generic.go:356] "Generic (PLEG): container finished" podID="be4a2511be029007da6e29235975f6dc" containerID="2ca236d176a2f93e30058df56a4f79b2ce18a552a651a3f3b284b12934e52060" exitCode=0
Nov 02 21:33:54.435478 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:54.434905    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerDied","Data":"2ca236d176a2f93e30058df56a4f79b2ce18a552a651a3f3b284b12934e52060"}
Nov 02 21:33:54.449734 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:54.449706    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"8ed0968c21495795a0292b52f44b9b55","Type":"ContainerStarted","Data":"9c60796e3404840c6286b30b3a377b62d11c14488f93aad116dd10e6c9562be8"}
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.034533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.034575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.034742    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.034804    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:59.034786527 +0000 UTC m=+10.094983145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.034898    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:55.035017 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.034936    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:33:59.034924889 +0000 UTC m=+10.095121503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:55.236833 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.236748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:55.248994 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.245431    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:33:55.248994 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.245459    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:33:55.248994 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.245473    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:55.248994 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.245541    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:33:59.245519676 +0000 UTC m=+10.305716310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:55.404576 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.402060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:55.404576 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.402184    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:33:55.404576 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.402257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:55.404576 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:55.402320    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:33:55.458516 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.458475    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerStarted","Data":"0028982bd254904f355007224b3dea79916abb577735a30022f129d384ce969a"}
Nov 02 21:33:55.463042 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.463012    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="1bc73d2b91bdd1ca39cbbc99659d3a36f18055aab7fba336c787feeb5169aac0" exitCode=0
Nov 02 21:33:55.463157 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.463115    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"1bc73d2b91bdd1ca39cbbc99659d3a36f18055aab7fba336c787feeb5169aac0"}
Nov 02 21:33:55.573530 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.572849    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal" podStartSLOduration=4.57283055 podStartE2EDuration="4.57283055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:54.661851225 +0000 UTC m=+5.722047859" watchObservedRunningTime="2026-11-02 21:33:55.57283055 +0000 UTC m=+6.633027184"
Nov 02 21:33:55.691100 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:55.690466    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" podStartSLOduration=4.690446155 podStartE2EDuration="4.690446155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:55.573322317 +0000 UTC m=+6.633518953" watchObservedRunningTime="2026-11-02 21:33:55.690446155 +0000 UTC m=+6.750642791"
Nov 02 21:33:56.401845 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:56.401782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:56.401983 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:56.401895    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:33:56.469971 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:56.469874    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wc9ll" event={"ID":"ab35d4ec-3432-43b7-adca-34097c507ae1","Type":"ContainerStarted","Data":"26cc45a115399d32ad1eb331d4bfb823473e91460eec29f275a5f3bf68d8ec5a"}
Nov 02 21:33:57.404303 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:57.404277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:57.404435 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:57.404401    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:33:57.404500 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:57.404462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:57.404542 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:57.404525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:33:57.478819 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:57.478743    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c65l7" event={"ID":"4905999a-0718-4c3d-8cfd-abe292d5e01c","Type":"ContainerStarted","Data":"130f6e448fca46dc10256e7472e22455287cb7517403fd0387d89cea37e6cea0"}
Nov 02 21:33:57.591846 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:57.590938    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wc9ll" podStartSLOduration=4.393644159 podStartE2EDuration="8.59091932s" totalImagesPullingTime="4.197275161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:56.584491321 +0000 UTC m=+7.644687954" watchObservedRunningTime="2026-11-02 21:33:57.59091932 +0000 UTC m=+8.651115956"
Nov 02 21:33:58.401562 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:58.401531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:58.401750 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:58.401643    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:33:58.483318 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:58.483223    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc" event={"ID":"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9","Type":"ContainerStarted","Data":"5b80a7776cca757094e078891962b25aaa61f956eff492e38d1da1aae92cf0cd"}
Nov 02 21:33:58.917688 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:58.917653    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:58.918761 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:58.918739    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:59.033080 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.033021    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c65l7" podStartSLOduration=4.510940712 podStartE2EDuration="10.033001581s" totalImagesPullingTime="5.522060869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:33:57.591813628 +0000 UTC m=+8.652010260" watchObservedRunningTime="2026-11-02 21:33:59.033001581 +0000 UTC m=+10.093198257"
Nov 02 21:33:59.070086 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.070054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:59.070086 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.070089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:33:59.070270 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.070187    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:59.070270 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.070233    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:07.070220569 +0000 UTC m=+18.130417180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:33:59.070358 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.070344    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:59.070405 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.070374    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:34:07.070367541 +0000 UTC m=+18.130564152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:33:59.271593 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.271514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:59.271741 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.271718    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:33:59.271741 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.271736    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:33:59.271830 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.271748    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:59.271830 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.271802    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:07.271786264 +0000 UTC m=+18.331982892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:33:59.403709 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.402070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:33:59.403709 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.402198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:33:59.403709 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.403593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:33:59.403709 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:33:59.403666    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:33:59.486371 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.486341    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:59.486820 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.486769    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:33:59.893634 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:33:59.893604    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 21:34:00.357651 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.357487    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T21:33:59.893624728Z","UUID":"c341671d-4d5c-41cc-8d1c-337e2f9df685","Handler":null,"Name":"","Endpoint":""}
Nov 02 21:34:00.359395 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.359370    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 21:34:00.359395 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.359396    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 21:34:00.402127 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.402098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:00.402280 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:00.402228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:00.490036 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.489998    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rwxvq" event={"ID":"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c","Type":"ContainerStarted","Data":"69a069878e1e8b4ba9f813637d41717ced5eefb2209da8c0f133161a2c367203"}
Nov 02 21:34:00.492914 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.492886    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="d6b572e9558e6841ac16f6b625db533874763657f24faec651ed909c37634ad0" exitCode=0
Nov 02 21:34:00.493046 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.492915    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"d6b572e9558e6841ac16f6b625db533874763657f24faec651ed909c37634ad0"}
Nov 02 21:34:00.495033 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.495013    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hkp7r" event={"ID":"942b2063-cdbd-49a9-bb85-21053e8f15c6","Type":"ContainerStarted","Data":"89ccb3652e17ca84cc5672bdbb92b3a770a9b86b0ccc0a787a6f83442d0dacb7"}
Nov 02 21:34:00.497494 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.497471    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc" event={"ID":"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9","Type":"ContainerStarted","Data":"9259f64094ccbc60f00aa0b0db0dc8a2c1bba2ec81645bb32c94298a94d9d9a9"}
Nov 02 21:34:00.604190 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.604094    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-rwxvq" podStartSLOduration=3.349266434 podStartE2EDuration="11.604076942s" totalImagesPullingTime="8.254810508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:00.603487544 +0000 UTC m=+11.663684202" watchObservedRunningTime="2026-11-02 21:34:00.604076942 +0000 UTC m=+11.664273576"
Nov 02 21:34:00.836871 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:00.836805    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hkp7r" podStartSLOduration=3.860027139 podStartE2EDuration="11.836786078s" totalImagesPullingTime="7.976758939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:00.835800838 +0000 UTC m=+11.895997473" watchObservedRunningTime="2026-11-02 21:34:00.836786078 +0000 UTC m=+11.896982710"
Nov 02 21:34:01.402295 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.402125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:01.402425 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.402162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:01.402425 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:01.402387    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:01.402551 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:01.402478    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:01.502239 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.502168    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vkrw9" event={"ID":"b35ac1d5-682e-4eab-9a5b-a0675b542860","Type":"ContainerStarted","Data":"0557a95925acf695f16fe00ba03fcea69b6aac65a63b4ed58a5a6d8a6e7eb520"}
Nov 02 21:34:01.506686 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.506659    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc" event={"ID":"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9","Type":"ContainerStarted","Data":"6d6c0ca670c874d5f7c66f91db230fa33a62b71d09c8b0b05d4d7829d9f729ad"}
Nov 02 21:34:01.615507 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.614898    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vkrw9" podStartSLOduration=4.511774874 podStartE2EDuration="12.614879243s" totalImagesPullingTime="8.103104369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:01.614308708 +0000 UTC m=+12.674505340" watchObservedRunningTime="2026-11-02 21:34:01.614879243 +0000 UTC m=+12.675075876"
Nov 02 21:34:01.729536 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:01.729478    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc" podStartSLOduration=3.559564342 podStartE2EDuration="12.72946351s" totalImagesPullingTime="9.169899168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:01.728836725 +0000 UTC m=+12.789033358" watchObservedRunningTime="2026-11-02 21:34:01.72946351 +0000 UTC m=+12.789660143"
Nov 02 21:34:02.401130 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:02.401093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:02.401303 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:02.401223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:02.509832 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:02.509793    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="60de31f65695e19341d357aab3f320c7bc9ccc2d20996dbd571e23c501456b93" exitCode=0
Nov 02 21:34:02.510418 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:02.509847    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"60de31f65695e19341d357aab3f320c7bc9ccc2d20996dbd571e23c501456b93"}
Nov 02 21:34:02.511655 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:02.511633    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pbvpg" event={"ID":"103a29ef-eca9-4925-93df-023ee970d7f7","Type":"ContainerStarted","Data":"5c9bc6f1ee1dadcec4e4b883a99ae40fed2777fb46e30c3c663db864fcb02264"}
Nov 02 21:34:02.741554 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:02.741458    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pbvpg" podStartSLOduration=3.582955149 podStartE2EDuration="13.741440639s" totalImagesPullingTime="10.15848549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:02.741033976 +0000 UTC m=+13.801230609" watchObservedRunningTime="2026-11-02 21:34:02.741440639 +0000 UTC m=+13.801637275"
Nov 02 21:34:03.401737 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:03.401706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:03.401911 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:03.401878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:03.401988 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:03.401898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:03.402038 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:03.402009    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:04.401724 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:04.401688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:04.402433 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:04.401832    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:04.519090 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:04.519057    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="a86e0011a02fc4382475f413ac52099d04b82e9cbc6ec2511356b506d463b693" exitCode=0
Nov 02 21:34:04.519251 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:04.519121    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"a86e0011a02fc4382475f413ac52099d04b82e9cbc6ec2511356b506d463b693"}
Nov 02 21:34:05.403752 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:05.403720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:05.404214 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:05.403828    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:05.404214 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:05.403891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:05.404214 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:05.404021    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:06.401387 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.401201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:06.401524 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:06.401503    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:06.526619 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.526595    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:34:06.527028 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.526914    2580 generic.go:356] "Generic (PLEG): container finished" podID="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" containerID="e15f7b47f687f48611670da126da4aa208d87cdaf9dcaf0552f01a971e25c92f" exitCode=1
Nov 02 21:34:06.527028 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.526981    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"c66cc592397effb5992f824398cb049798edb55cba1f504b9a0753dfb37c762d"}
Nov 02 21:34:06.527101 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.527035    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"460585c78aaf6fe83d4b90496f2a99aa8ef41ce681f454ff1dc5568e5bd076a1"}
Nov 02 21:34:06.527101 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.527064    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"dbeef9ea0db413fdf638ef259f678b744df969e5ebc9b6b7210fb0b457d83a74"}
Nov 02 21:34:06.527101 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.527090    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"8bc2da7f4e3d7dfe557c90d76148d72549a071078ca0861584e43eaa6eace3db"}
Nov 02 21:34:06.527195 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.527115    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerDied","Data":"e15f7b47f687f48611670da126da4aa208d87cdaf9dcaf0552f01a971e25c92f"}
Nov 02 21:34:06.527195 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:06.527144    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"6930088d26fbc686da258bcb1a5f9216ef3a2460eb7c1969a42c598017a01a93"}
Nov 02 21:34:07.139703 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.139634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:07.139703 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.139666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:07.139899 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.139776    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:34:07.139899 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.139786    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:34:07.139899 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.139825    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:23.139813847 +0000 UTC m=+34.200010471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:34:07.139899 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.139837    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:34:23.139831893 +0000 UTC m=+34.200028504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:34:07.341560 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.341536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:07.341688 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.341643    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:34:07.341688 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.341657    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:34:07.341688 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.341674    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:34:07.341787 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.341715    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:23.341704651 +0000 UTC m=+34.401901267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:34:07.401815 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.401746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:07.401936 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.401746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:07.401936 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.401858    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:07.401936 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:07.401908    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:07.531699 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.531666    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="2f3e2cb54fbbd371854f27e1051579db33d52ee320e9dac1308b53af859139b6" exitCode=0
Nov 02 21:34:07.532295 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:07.531731    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"2f3e2cb54fbbd371854f27e1051579db33d52ee320e9dac1308b53af859139b6"}
Nov 02 21:34:08.401898 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:08.401860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:08.402083 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:08.401978    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:08.537505 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:08.537478    2580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="49a748c1d4fa17981fd20577db47e8547d5862e394a2dc4c902dcc2d8750422f" exitCode=0
Nov 02 21:34:08.537866 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:08.537521    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerDied","Data":"49a748c1d4fa17981fd20577db47e8547d5862e394a2dc4c902dcc2d8750422f"}
Nov 02 21:34:09.402537 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.402497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:09.402714 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:09.402620    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:09.402714 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.402671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:09.402845 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:09.402738    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:09.543174 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.543141    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" event={"ID":"e88de0d5-7833-4038-8119-201c5fff01f5","Type":"ContainerStarted","Data":"2f5f3d629fefa1550e0b5a16d0fd934cafd3f1127bf47531ac6ea2e7de6cfdfe"}
Nov 02 21:34:09.546110 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.546090    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:34:09.546434 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.546415    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"88e7aa209fa039b4735eeb5902c579d14176e053fdaec3fbc273eba86f03e65d"}
Nov 02 21:34:09.660412 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:09.660324    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dcf5t" podStartSLOduration=8.919825046 podStartE2EDuration="20.660311479s" totalImagesPullingTime="11.740486433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:09.660124814 +0000 UTC m=+20.720321446" watchObservedRunningTime="2026-11-02 21:34:09.660311479 +0000 UTC m=+20.720508111"
Nov 02 21:34:10.401180 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:10.401143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:10.403726 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:10.401642    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:11.401556 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.401376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:11.402090 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.401412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:11.402090 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:11.401629    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:11.402090 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:11.401746    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:11.553334 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.553307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:34:11.553709 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.553681    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"9c79bd99adee9899bd8da3543ae3ec260f8a858a339275215796659a1006fdc0"}
Nov 02 21:34:11.553937 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.553920    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:11.554133 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.554118    2580 scope.go:122] "RemoveContainer" containerID="e15f7b47f687f48611670da126da4aa208d87cdaf9dcaf0552f01a971e25c92f"
Nov 02 21:34:11.571846 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:11.571825    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:12.401301 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.401273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:12.401482 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:12.401383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:12.559658 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.559630    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:34:12.560037 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.559947    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" event={"ID":"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7","Type":"ContainerStarted","Data":"6e10d0e9c60f100b558213f3cb859495300b3e47af0e4b0bdefc3edbcbd227a8"}
Nov 02 21:34:12.560087 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.560044    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 21:34:12.560303 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.560281    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:12.574431 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.574409    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:12.738243 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.738158    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c" podStartSLOduration=9.572589028 podStartE2EDuration="23.738145933s" totalImagesPullingTime="14.165556905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:12.737345836 +0000 UTC m=+23.797542471" watchObservedRunningTime="2026-11-02 21:34:12.738145933 +0000 UTC m=+23.798342566"
Nov 02 21:34:12.814117 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:12.814095    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rwxvq_78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c/dns-node-resolver/0.log"
Nov 02 21:34:13.094942 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:13.094866    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:13.401137 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:13.401074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:13.401289 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:13.401175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:13.401289 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:13.401166    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:13.401289 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:13.401275    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:13.642005 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:13.641983    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c65l7_4905999a-0718-4c3d-8cfd-abe292d5e01c/node-ca/0.log"
Nov 02 21:34:14.402018 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:14.401988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:14.402178 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:14.402112    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:15.401677 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:15.401647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:15.402120 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:15.401746    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:15.402120 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:15.401831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:15.402120 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:15.401938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:16.401725 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:16.401697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:16.402108 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:16.401796    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:17.401283 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:17.401252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:17.401581 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:17.401356    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:17.401581 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:17.401418    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:17.401581 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:17.401528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:18.402159 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:18.402069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:18.402518 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:18.402204    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:19.401974 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:19.401935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:19.402155 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:19.402084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:19.402155 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:19.402125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:19.402467 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:19.402181    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:20.401476 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:20.401450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:20.401630 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:20.401545    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:21.051517 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:21.051485    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wc9ll_ab35d4ec-3432-43b7-adca-34097c507ae1/konnectivity-agent/0.log" limit=1024
Nov 02 21:34:21.401382 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:21.401303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:21.401532 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:21.401309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:21.401532 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:21.401426    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:21.401532 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:21.401511    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:21.484874 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:21.484830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wc9ll_ab35d4ec-3432-43b7-adca-34097c507ae1/konnectivity-agent/0.log" limit=1024
Nov 02 21:34:21.687651 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:21.687620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wc9ll_ab35d4ec-3432-43b7-adca-34097c507ae1/konnectivity-agent/0.log" limit=1024
Nov 02 21:34:22.401620 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:22.401586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:22.402131 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:22.401728    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:23.152417 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:23.152388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:23.152417 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:23.152420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:23.152615 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.152541    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 21:34:23.152615 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.152593    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:34:23.152615 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.152602    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret podName:2e177cbc-5f97-45c8-9714-73510ec5597b nodeName:}" failed. No retries permitted until 2026-11-02 21:34:55.152587546 +0000 UTC m=+66.212784158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret") pod "global-pull-secret-syncer-ljkpr" (UID: "2e177cbc-5f97-45c8-9714-73510ec5597b") : object "kube-system"/"original-pull-secret" not registered
Nov 02 21:34:23.152711 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.152632    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs podName:3c18553f-6a3e-4c2a-a4e1-424c05f27216 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:55.152620884 +0000 UTC m=+66.212817499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs") pod "network-metrics-daemon-x2887" (UID: "3c18553f-6a3e-4c2a-a4e1-424c05f27216") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 21:34:23.353640 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:23.353610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:23.353821 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.353767    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 21:34:23.353821 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.353786    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 21:34:23.353821 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.353796    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zj598 for pod openshift-network-diagnostics/network-check-target-7t4gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:34:23.354017 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.353853    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598 podName:2f254a4a-36cb-4152-8c7e-b9a1c441c6d8 nodeName:}" failed. No retries permitted until 2026-11-02 21:34:55.353835325 +0000 UTC m=+66.414031938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zj598" (UniqueName: "kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598") pod "network-check-target-7t4gd" (UID: "2f254a4a-36cb-4152-8c7e-b9a1c441c6d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 21:34:23.402151 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:23.402117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:23.402701 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.402223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:23.402701 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:23.402254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:23.402701 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:23.402301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:24.401816 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:24.401784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:24.401990 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:24.401892    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:25.401437 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:25.401406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:25.401888 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:25.401406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:25.401888 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:25.401525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:25.401888 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:25.401565    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:26.401512 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:26.401486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:26.401901 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:26.401581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:27.401724 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:27.401687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:27.402183 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:27.401818    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:27.402183 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:27.401860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:27.402183 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:27.401943    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:28.401424 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:28.401391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:28.401590 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:28.401505    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:29.404726 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:29.404699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:29.405108 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:29.404739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:29.405108 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:29.404816    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:29.405108 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:29.404877    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:30.401178 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:30.401138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:30.401349 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:30.401249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:31.401725 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:31.401684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:31.402224 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:31.401734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:31.402224 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:31.401797    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:31.402224 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:31.401931    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:32.375573 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.375325    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljkpr"]
Nov 02 21:34:32.375725 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.375672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:32.375801 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:32.375781    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:32.378297 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.378270    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7t4gd"]
Nov 02 21:34:32.378439 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.378381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:32.378505 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:32.378479    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:32.379138 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.379112    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2887"]
Nov 02 21:34:32.379226 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:32.379213    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:32.379347 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:32.379327    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:34.401932 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:34.401896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:34.402425 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:34.401896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:34.402425 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:34.402018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:34.402425 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:34.402053    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:34.402425 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:34.402165    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:34.402425 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:34.402245    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:36.401377 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:36.401350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:36.401665 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:36.401355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:36.401665 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:36.401453    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7t4gd" podUID="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8"
Nov 02 21:34:36.401665 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:36.401368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:36.401665 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:36.401516    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkpr" podUID="2e177cbc-5f97-45c8-9714-73510ec5597b"
Nov 02 21:34:36.401665 ip-10-0-136-74 kubenswrapper[2580]: E1102 21:34:36.401600    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2887" podUID="3c18553f-6a3e-4c2a-a4e1-424c05f27216"
Nov 02 21:34:37.766460 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.766434    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeReady"
Nov 02 21:34:37.766835 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.766553    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 21:34:37.931196 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.931169    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qw24r"]
Nov 02 21:34:37.936771 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.936745    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9hsfk"]
Nov 02 21:34:37.936913 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.936859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:37.938567 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.938539    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xht6q"]
Nov 02 21:34:37.938688 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.938651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:37.940398 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.940378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:37.988861 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.988830    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qw24r"]
Nov 02 21:34:37.991829 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.991808    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 21:34:37.991938 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.991884    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-plzkt\""
Nov 02 21:34:37.992019 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992008    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 21:34:37.992062 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992051    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 21:34:37.992124 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992113    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 21:34:37.992245 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992232    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-stcz5\""
Nov 02 21:34:37.992302 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992277    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 21:34:37.992359 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992346    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 21:34:37.992405 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:37.992359    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hsfk"]
Nov 02 21:34:38.039106 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.039015    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 21:34:38.039106 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.039018    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 21:34:38.039106 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.039056    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5rwnh\""
Nov 02 21:34:38.039106 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.039057    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 21:34:38.050079 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.050055    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xht6q"]
Nov 02 21:34:38.054847 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.054817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.055001 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.054866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.055001 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.054890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.055121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.054994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvs7m\" (UniqueName: \"kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.055121 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.055220 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59sw4\" (UniqueName: \"kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.055220 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.055286 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.055286 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.055351 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.055446 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.055433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4268\" (UniqueName: \"kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.156471 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.156471 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4268\" (UniqueName: \"kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvs7m\" (UniqueName: \"kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.156715 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sw4\" (UniqueName: \"kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.157148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.156743    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.157230 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.157203    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.157282 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.157206    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.157326 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.157313    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.157360 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.157336    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.160368 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.160346    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.160467 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.160393    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.160467 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.160433    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.213764 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.213726    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sw4\" (UniqueName: \"kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.213929 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.213830    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4268\" (UniqueName: \"kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.213929 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.213906    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvs7m\" (UniqueName: \"kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.250241 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.250211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:38.256067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.256043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:34:38.258616 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.258589    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:34:38.401894 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.401853    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:38.401894 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.401882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:38.402138 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.401892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:38.455113 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455072    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 21:34:38.455299 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455285    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 21:34:38.455457 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455439    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-fbz7r\""
Nov 02 21:34:38.455504 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455439    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 21:34:38.455504 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455459    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 21:34:38.455504 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455467    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zd85x\""
Nov 02 21:34:38.455595 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.455494    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jhcsj\""
Nov 02 21:34:38.633595 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.633509    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qw24r"]
Nov 02 21:34:38.636341 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:38.636310    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3fb92a_6bfe_4d2f_968e_81741b324547.slice/crio-ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b WatchSource:0}: Error finding container ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b: Status 404 returned error can't find the container with id ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b
Nov 02 21:34:38.677292 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.677255    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xht6q"]
Nov 02 21:34:38.681514 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:38.680181    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hsfk"]
Nov 02 21:34:38.683063 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:38.682986    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec2dacc_d493_48f3_9bec_45e232e24e4c.slice/crio-6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb WatchSource:0}: Error finding container 6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb: Status 404 returned error can't find the container with id 6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb
Nov 02 21:34:38.686074 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:38.686055    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d83a7a1_7653_4b92_ada5_137be7518e4e.slice/crio-70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787 WatchSource:0}: Error finding container 70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787: Status 404 returned error can't find the container with id 70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787
Nov 02 21:34:39.619265 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:39.619214    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hsfk" event={"ID":"7d83a7a1-7653-4b92-ada5-137be7518e4e","Type":"ContainerStarted","Data":"c7b1b3c5dcbeb7c6fdf13d16c58cc7d6a83c957013a190bd574fe35dcc145842"}
Nov 02 21:34:39.619703 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:39.619296    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hsfk" event={"ID":"7d83a7a1-7653-4b92-ada5-137be7518e4e","Type":"ContainerStarted","Data":"70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787"}
Nov 02 21:34:39.620551 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:39.620513    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qw24r" event={"ID":"1b3fb92a-6bfe-4d2f-968e-81741b324547","Type":"ContainerStarted","Data":"ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b"}
Nov 02 21:34:39.622074 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:39.622050    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xht6q" event={"ID":"aec2dacc-d493-48f3-9bec-45e232e24e4c","Type":"ContainerStarted","Data":"6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb"}
Nov 02 21:34:40.347094 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.347009    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"]
Nov 02 21:34:40.349204 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.349180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.404317 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.404280    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 02 21:34:40.405800 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.405769    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"]
Nov 02 21:34:40.455280 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.455253    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 02 21:34:40.455486 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.455300    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fwwgs\""
Nov 02 21:34:40.455486 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.455354    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 21:34:40.455621 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.455493    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 21:34:40.455621 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.455583    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 21:34:40.475004 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.474948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.475168 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.475020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.475245 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.475163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p96zw\" (UniqueName: \"kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.475245 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.475224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.507106 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.507064    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"]
Nov 02 21:34:40.509479 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.509449    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gg8vl"]
Nov 02 21:34:40.509616 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.509597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.511668 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.511646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.524152 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.524069    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"]
Nov 02 21:34:40.576178 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.576133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.576374 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.576219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p96zw\" (UniqueName: \"kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.576374 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.576263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.576374 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.576324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.577107 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.577081    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.579362 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.579334    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.579567 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.579543    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.625209 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.625131    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.625127    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630046    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630316    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-z597s\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630381    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xsz5x\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630486    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630493    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 21:34:40.631911 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.630592    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 02 21:34:40.635567 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.635541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p96zw\" (UniqueName: \"kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.635717 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.635661    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hsfk" event={"ID":"7d83a7a1-7653-4b92-ada5-137be7518e4e","Type":"ContainerStarted","Data":"d13ecf0bcec4cba25e6dbbcc7dfc934e63ac0c7a7c443aa2a19bf4747132a8c1"}
Nov 02 21:34:40.664048 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.664011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.677844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdfv\" (UniqueName: \"kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxgq\" (UniqueName: \"kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.681612 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.678310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdfv\" (UniqueName: \"kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxgq\" (UniqueName: \"kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.778974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.782572 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.779309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.780172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.780758    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.781249    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.781467    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.781906    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.782040    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.782177    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.782511    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.783461 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.782882    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.783870 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.783658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.784215 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.784194    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.784366 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.784346    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:40.837195 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.837162    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdfv\" (UniqueName: \"kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:40.837609 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:40.837584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxgq\" (UniqueName: \"kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:41.092973 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.092930    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"]
Nov 02 21:34:41.095312 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:41.095268    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f84beb_14a1_406c_924e_3896b29f6564.slice/crio-d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c WatchSource:0}: Error finding container d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c: Status 404 returned error can't find the container with id d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c
Nov 02 21:34:41.127308 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.127282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:34:41.133285 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.133264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:34:41.566952 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.566921    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"]
Nov 02 21:34:41.640491 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.640406    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gg8vl" event={"ID":"459afbc7-52e2-42f9-98f0-b157dfc74671","Type":"ContainerStarted","Data":"3fe9687f35d72796591c0499551d12a4244a2030b407969df266d8b423f83e44"}
Nov 02 21:34:41.643005 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.642979    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qw24r" event={"ID":"1b3fb92a-6bfe-4d2f-968e-81741b324547","Type":"ContainerStarted","Data":"a582d4f75d37efe4e91050143a98c636b3037c8fd1f09be8dd0f79289e840c25"}
Nov 02 21:34:41.643138 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.643034    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qw24r" event={"ID":"1b3fb92a-6bfe-4d2f-968e-81741b324547","Type":"ContainerStarted","Data":"88b6e9748d76955c9f3e0efc0a33a843a7bad66e2f927b06a1403791ba70fd67"}
Nov 02 21:34:41.643138 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.643103    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:41.644480 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.644459    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b" event={"ID":"4e1a1df8-8a6a-4136-9444-9eb795619aed","Type":"ContainerStarted","Data":"464a3c3d5b4c39b207b152bffd919fc9c2acf949cd8d5993d86d43f135c41391"}
Nov 02 21:34:41.646699 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.646679    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l" event={"ID":"a3f84beb-14a1-406c-924e-3896b29f6564","Type":"ContainerStarted","Data":"0dda69b9743e4c681caf2ef228b775673a1baa13dfc8036f988a12592d9ca321"}
Nov 02 21:34:41.646801 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.646722    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l" event={"ID":"a3f84beb-14a1-406c-924e-3896b29f6564","Type":"ContainerStarted","Data":"996987cddd188d05438f528674c2243af1c527d3fd8e162dab5f393f78613c0c"}
Nov 02 21:34:41.646801 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.646745    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l" event={"ID":"a3f84beb-14a1-406c-924e-3896b29f6564","Type":"ContainerStarted","Data":"d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c"}
Nov 02 21:34:41.758873 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:41.758809    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-qw24r" podStartSLOduration=2.384971039 podStartE2EDuration="4.758791537s" totalImagesPullingTime="2.373820498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:41.757891801 +0000 UTC m=+52.818088468" watchObservedRunningTime="2026-11-02 21:34:41.758791537 +0000 UTC m=+52.818988171"
Nov 02 21:34:42.652813 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.652776    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xht6q" event={"ID":"aec2dacc-d493-48f3-9bec-45e232e24e4c","Type":"ContainerStarted","Data":"f8d7348dbe264da93c0d90e3a564c5dd0ceff62e327b8fa695bf81bdf357cc25"}
Nov 02 21:34:42.659426 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.658982    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hsfk" event={"ID":"7d83a7a1-7653-4b92-ada5-137be7518e4e","Type":"ContainerStarted","Data":"c82b6b4252a87d43e8507781b400fd99e15a4f2a3bf5170fe9393d6bb2064951"}
Nov 02 21:34:42.767738 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.767676    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xht6q" podStartSLOduration=2.583651846 podStartE2EDuration="5.76765905s" totalImagesPullingTime="3.184007204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:42.766811542 +0000 UTC m=+53.827008175" watchObservedRunningTime="2026-11-02 21:34:42.76765905 +0000 UTC m=+53.827855690"
Nov 02 21:34:42.883065 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.882971    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9hsfk" podStartSLOduration=2.621293326 podStartE2EDuration="5.882941614s" totalImagesPullingTime="3.261648288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 21:34:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:42.881796847 +0000 UTC m=+53.941993507" watchObservedRunningTime="2026-11-02 21:34:42.882941614 +0000 UTC m=+53.943138247"
Nov 02 21:34:42.936364 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.936329    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"]
Nov 02 21:34:42.939104 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.939081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:42.992265 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992233    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 02 21:34:42.992265 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992265    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 02 21:34:42.992265 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992261    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 02 21:34:42.992493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992281    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 02 21:34:42.992493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992343    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7gg4d807kkhps\""
Nov 02 21:34:42.992493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992362    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q2lsm\""
Nov 02 21:34:42.992493 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.992409    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 02 21:34:42.998253 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:42.998227    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"]
Nov 02 21:34:43.101072 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101331 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101331 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101331 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101331 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjrhh\" (UniqueName: \"kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.101662 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.101541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.202842 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.202801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.202865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.202895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.202918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjrhh\" (UniqueName: \"kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.202977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203067 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203314 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203314 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203438 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203413    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.203675 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.203651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.205974 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.205937    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.206219 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.206193    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.206464 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.206446    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.206854 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.206831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.207403 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.207378    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.207489 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.207440    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.260912 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.260878    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjrhh\" (UniqueName: \"kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.556303 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.556270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:43.666634 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.666596    2580 generic.go:356] "Generic (PLEG): container finished" podID="459afbc7-52e2-42f9-98f0-b157dfc74671" containerID="603ff591fe76e02e5e0e906cbb9c4c32773c8df7ee8fe271fa7ae31783df8f0c" exitCode=0
Nov 02 21:34:43.667056 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.666645    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gg8vl" event={"ID":"459afbc7-52e2-42f9-98f0-b157dfc74671","Type":"ContainerDied","Data":"603ff591fe76e02e5e0e906cbb9c4c32773c8df7ee8fe271fa7ae31783df8f0c"}
Nov 02 21:34:43.668873 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.668854    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b" event={"ID":"4e1a1df8-8a6a-4136-9444-9eb795619aed","Type":"ContainerStarted","Data":"fe0307e7137dd656871119f9fb24358f5edb669fafce55d1986dd4d642073b42"}
Nov 02 21:34:43.668999 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.668900    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b" event={"ID":"4e1a1df8-8a6a-4136-9444-9eb795619aed","Type":"ContainerStarted","Data":"de69c3ae525f138feb4b0c232742626186cfdc2c51ec862bd4bc72ef2a41e57a"}
Nov 02 21:34:43.668999 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.668926    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b" event={"ID":"4e1a1df8-8a6a-4136-9444-9eb795619aed","Type":"ContainerStarted","Data":"f2248d579b1e17c4ac0e0ee165b3420df988611d487feaeab2b1b3107132b820"}
Nov 02 21:34:43.671044 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.671019    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l" event={"ID":"a3f84beb-14a1-406c-924e-3896b29f6564","Type":"ContainerStarted","Data":"c9e5060c6c8f24b4b2c3051655f0fe503d944a03c1b3527ec08073c8331d5664"}
Nov 02 21:34:43.901638 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.901588    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l" podStartSLOduration=2.469714105 podStartE2EDuration="3.901573342s" totalImagesPullingTime="1.431859237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:43.901443415 +0000 UTC m=+54.961640049" watchObservedRunningTime="2026-11-02 21:34:43.901573342 +0000 UTC m=+54.961769977"
Nov 02 21:34:43.973636 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:43.973612    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"]
Nov 02 21:34:43.975875 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:43.975841    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be53397_eac6_47b3_8b63_faa5479e1f3b.slice/crio-339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76 WatchSource:0}: Error finding container 339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76: Status 404 returned error can't find the container with id 339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76
Nov 02 21:34:44.021206 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.021159    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b" podStartSLOduration=2.244076002 podStartE2EDuration="4.021145809s" totalImagesPullingTime="1.777069807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:44.020679264 +0000 UTC m=+55.080875898" watchObservedRunningTime="2026-11-02 21:34:44.021145809 +0000 UTC m=+55.081342441"
Nov 02 21:34:44.574806 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.574780    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:34:44.676255 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.676216    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gg8vl" event={"ID":"459afbc7-52e2-42f9-98f0-b157dfc74671","Type":"ContainerStarted","Data":"750907a9b12986a7f230f95040c6ff851825751def4bf2dfa4b042cf1f9ae55a"}
Nov 02 21:34:44.676689 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.676286    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gg8vl" event={"ID":"459afbc7-52e2-42f9-98f0-b157dfc74671","Type":"ContainerStarted","Data":"2564e6b34e5d9257328f733e77039d955880946ed27a05e139880e9820a01b12"}
Nov 02 21:34:44.677273 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.677252    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76"}
Nov 02 21:34:44.867326 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:44.867234    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gg8vl" podStartSLOduration=3.36301464 podStartE2EDuration="4.867219427s" totalImagesPullingTime="1.424023195s" totalInitContainerRuntime="80.181592ms" isStatefulPod=true podCreationTimestamp="2026-11-02 21:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:44.866303043 +0000 UTC m=+55.926499693" watchObservedRunningTime="2026-11-02 21:34:44.867219427 +0000 UTC m=+55.927416059"
Nov 02 21:34:45.082332 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.082297    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"]
Nov 02 21:34:45.084322 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.084302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.118347 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.118281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.118474 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.118456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j452n\" (UniqueName: \"kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.140822 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.140795    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"]
Nov 02 21:34:45.185204 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.185174    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 02 21:34:45.185204 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.185174    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6b6c5\""
Nov 02 21:34:45.220173 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.220136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.220321 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.220294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j452n\" (UniqueName: \"kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.226491 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.226463    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.278314 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.278280    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j452n\" (UniqueName: \"kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.396163 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.396084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:45.766741 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:45.766713    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"]
Nov 02 21:34:45.770302 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:45.770265    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58fab09_a42f_4f63_96a9_cfe58f7ed15c.slice/crio-b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437 WatchSource:0}: Error finding container b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437: Status 404 returned error can't find the container with id b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437
Nov 02 21:34:46.685523 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:46.685483    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt" event={"ID":"e58fab09-a42f-4f63-96a9-cfe58f7ed15c","Type":"ContainerStarted","Data":"b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437"}
Nov 02 21:34:46.687849 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:46.687820    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"96adf204e350f686086de98f68dcfdc3bcb43064e9a3f1e2d9856d1c1732a6c9"}
Nov 02 21:34:46.687950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:46.687893    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"39f5a853ed3de3b21a8860491daeded6e5880da5cdb1e97c61dc0c3783993426"}
Nov 02 21:34:46.687950 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:46.687932    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"1c0fb78ab975510bd1d64b37f1ff041cf2d1a0a721062e4b7d7774aabcde9cad"}
Nov 02 21:34:47.694667 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:47.694638    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt" event={"ID":"e58fab09-a42f-4f63-96a9-cfe58f7ed15c","Type":"ContainerStarted","Data":"863c22bef4e43e8a4ccc3ce335f7d3421d6b71513b64f1afd3602c1404dcc633"}
Nov 02 21:34:47.695049 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:47.694704    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:47.697352 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:47.697331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"2f661796ab9a013a5f596ec7f85373cd95c7f65eeb6ba34782b33d1730944336"}
Nov 02 21:34:47.699848 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:47.699819    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:34:47.807696 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:47.807649    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt" podStartSLOduration=1.119993321 podStartE2EDuration="2.807636199s" totalImagesPullingTime="1.687642878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:47.807328672 +0000 UTC m=+58.867525304" watchObservedRunningTime="2026-11-02 21:34:47.807636199 +0000 UTC m=+58.867832828"
Nov 02 21:34:48.703919 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:48.703886    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"189aaea0311cb72b27a2f79ec16e3ca852eba3e75b6502fb54e454b3ad2a5052"}
Nov 02 21:34:48.704283 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:48.703973    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" event={"ID":"3be53397-eac6-47b3-8b63-faa5479e1f3b","Type":"ContainerStarted","Data":"3978a7f3b662ae4711e1a98c26bde9ee216ca244969fea97a18c63a85dad9cf8"}
Nov 02 21:34:48.924352 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:48.924302    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" podStartSLOduration=3.459640156 podStartE2EDuration="6.924284925s" totalImagesPullingTime="3.464644769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:48.873718685 +0000 UTC m=+59.933915318" watchObservedRunningTime="2026-11-02 21:34:48.924284925 +0000 UTC m=+59.984481551"
Nov 02 21:34:49.257323 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.257291    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:34:49.260029 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.260011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.313465 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.313425    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 02 21:34:49.324643 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.324618    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-51ss0e74f2ivr\""
Nov 02 21:34:49.324780 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.324659    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 02 21:34:49.324780 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.324686    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 02 21:34:49.324780 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.324685    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 02 21:34:49.324780 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.324699    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bzpj6\""
Nov 02 21:34:49.356170 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356170 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356337 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356337 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356406 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356406 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356406 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356650 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356650 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356712 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356712 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356795 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356795 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356887 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356887 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.356887 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.356879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.359912 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359890    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 02 21:34:49.359912 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359903    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 02 21:34:49.359912 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359890    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 02 21:34:49.360102 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359909    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 02 21:34:49.360102 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359918    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 02 21:34:49.360102 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.359993    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 02 21:34:49.368207 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.368185    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:34:49.414211 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.414187    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 02 21:34:49.414343 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.414246    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 02 21:34:49.414343 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.414251    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 02 21:34:49.457940 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.457914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.457940 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.457942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.457978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.457997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458148 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458496 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458905 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458848    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458905 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458865    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.458905 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458889    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.459069 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.458947    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.459468 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.459444    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.461220 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461188    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.461748 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461439    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.461748 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.461748 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.461748 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461612    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.462018 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.461926    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.462160 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.462129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.462324 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.462301    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.463873 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.463843    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.463975 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.463852    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.463975 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.463873    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.464318 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.464303    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.465667 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.465648    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 02 21:34:49.465738 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.465704    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 02 21:34:49.469196 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.469173    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.470674 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.470653    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.515578 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.515522    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.578733 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.578706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:34:49.707298 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:49.707273    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:50.002941 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:50.002919    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:34:50.004818 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:50.004789    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880e1a9f_5711_436e_938f_619f3e692a56.slice/crio-36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb WatchSource:0}: Error finding container 36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb: Status 404 returned error can't find the container with id 36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb
Nov 02 21:34:50.711794 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:50.711757    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb"}
Nov 02 21:34:51.661576 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:51.661502    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qw24r"
Nov 02 21:34:51.716331 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:51.716297    2580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b" exitCode=0
Nov 02 21:34:51.716670 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:51.716355    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"}
Nov 02 21:34:55.206182 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.206149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:55.206519 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.206207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:55.258462 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.258443    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 21:34:55.258534 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.258474    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 21:34:55.268814 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.268793    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:55.269035 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.269015    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:55.408147 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.408121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:55.460211 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.460174    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 21:34:55.521408 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.521383    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 21:34:55.534511 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.534453    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:55.567476 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.567447    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zd85x\""
Nov 02 21:34:55.575195 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.575172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:34:55.575195 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.575188    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-fbz7r\""
Nov 02 21:34:55.583274 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.583247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:34:55.718187 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.718106    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:34:55.732255 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732225    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"}
Nov 02 21:34:55.732363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732278    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"}
Nov 02 21:34:55.732363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732304    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"}
Nov 02 21:34:55.732363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732326    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"}
Nov 02 21:34:55.732363 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732347    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"}
Nov 02 21:34:55.732506 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.732368    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerStarted","Data":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"}
Nov 02 21:34:55.874400 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.874368    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jhcsj\""
Nov 02 21:34:55.879460 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.879439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:34:55.961830 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.961807    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2887"]
Nov 02 21:34:55.964632 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:55.964608    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c18553f_6a3e_4c2a_a4e1_424c05f27216.slice/crio-5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85 WatchSource:0}: Error finding container 5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85: Status 404 returned error can't find the container with id 5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85
Nov 02 21:34:55.976801 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:55.976728    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.117088615 podStartE2EDuration="6.976715262s" totalImagesPullingTime="4.845689584s" totalInitContainerRuntime="13.937063ms" isStatefulPod=false podCreationTimestamp="2026-11-02 21:34:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:55.975996577 +0000 UTC m=+67.036193208" watchObservedRunningTime="2026-11-02 21:34:55.976715262 +0000 UTC m=+67.036911938"
Nov 02 21:34:56.005998 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:56.005970    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljkpr"]
Nov 02 21:34:56.014669 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:56.014645    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e177cbc_5f97_45c8_9714_73510ec5597b.slice/crio-b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6 WatchSource:0}: Error finding container b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6: Status 404 returned error can't find the container with id b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6
Nov 02 21:34:56.249261 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:56.249018    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7t4gd"]
Nov 02 21:34:56.251597 ip-10-0-136-74 kubenswrapper[2580]: W1102 21:34:56.251569    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f254a4a_36cb_4152_8c7e_b9a1c441c6d8.slice/crio-f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef WatchSource:0}: Error finding container f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef: Status 404 returned error can't find the container with id f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef
Nov 02 21:34:56.736947 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:56.736907    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7t4gd" event={"ID":"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8","Type":"ContainerStarted","Data":"f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef"}
Nov 02 21:34:56.738007 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:56.737979    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ljkpr" event={"ID":"2e177cbc-5f97-45c8-9714-73510ec5597b","Type":"ContainerStarted","Data":"b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6"}
Nov 02 21:34:56.738998 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:56.738975    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2887" event={"ID":"3c18553f-6a3e-4c2a-a4e1-424c05f27216","Type":"ContainerStarted","Data":"5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85"}
Nov 02 21:34:57.750433 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:57.750297    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2887" event={"ID":"3c18553f-6a3e-4c2a-a4e1-424c05f27216","Type":"ContainerStarted","Data":"41b1660cdf689e83a4f3ff123c40ef8f183dcc3d5b9d025a81d991970c1462ba"}
Nov 02 21:34:57.750857 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:57.750443    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2887" event={"ID":"3c18553f-6a3e-4c2a-a4e1-424c05f27216","Type":"ContainerStarted","Data":"261bd7577558eee9fee1dcfcd0f08554794616331c922d8841dc80a5d5fa7340"}
Nov 02 21:34:57.868090 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:57.868032    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x2887" podStartSLOduration=67.400789347 podStartE2EDuration="1m8.868014041s" totalImagesPullingTime="1.467224694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:34:57.867082097 +0000 UTC m=+68.927278730" watchObservedRunningTime="2026-11-02 21:34:57.868014041 +0000 UTC m=+68.928210674"
Nov 02 21:34:59.579731 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:34:59.579695    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:00.764669 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:35:00.764629    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7t4gd" event={"ID":"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8","Type":"ContainerStarted","Data":"73ed73f0bc9063c38c84da1a148dcca6e4b290a3dd72d4529045e80a099e5a90"}
Nov 02 21:35:00.765163 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:35:00.764880    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:35:00.879820 ip-10-0-136-74 kubenswrapper[2580]: I1102 21:35:00.879763    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7t4gd" podStartSLOduration=68.253032455 podStartE2EDuration="1m11.879749405s" totalImagesPullingTime="3.62671695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:00.878761895 +0000 UTC m=+71.938958529" watchObservedRunningTime="2026-11-02 21:35:00.879749405 +0000 UTC m=+71.939946044"
Nov 02 21:35:01.424579 ip-10-0-136-74 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 21:35:01.442381 ip-10-0-136-74 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 21:35:01.442538 ip-10-0-136-74 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 21:35:01.442584 ip-10-0-136-74 systemd[1]: kubelet.service: Consumed 4.075s CPU time.
Nov 02 21:35:01.454567 ip-10-0-136-74 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 21:35:01.513170 ip-10-0-136-74 kubenswrapper[5580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:35:01.513170 ip-10-0-136-74 kubenswrapper[5580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 21:35:01.513170 ip-10-0-136-74 kubenswrapper[5580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:35:01.513666 ip-10-0-136-74 kubenswrapper[5580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 21:35:01.516395 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516315    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.516395 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516392    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.516395 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516397    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.516395 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516400    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516403    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516407    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516410    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516412    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516415    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516417    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516420    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516423    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516426    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516428    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516431    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516433    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516436    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516439    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516444    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516447    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516450    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516455    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.516566 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516458    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516462    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516465    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516467    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516470    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516472    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516475    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516478    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516481    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516483    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516486    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516488    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516491    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516494    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516497    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516500    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516502    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516505    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516507    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516510    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.517019 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516512    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516515    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516517    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516520    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516522    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516525    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516527    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516530    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516540    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516542    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516545    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516547    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516550    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516553    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516555    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516558    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516561    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516563    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516566    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.517477 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516569    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516571    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516574    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516576    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516579    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516582    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516585    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516587    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516589    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516592    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516595    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516598    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516600    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516602    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516605    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516609    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516611    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516614    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516616    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516619    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.517918 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516621    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516623    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516632    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516635    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516637    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516640    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516642    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516647    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516649    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516652    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516655    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516658    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516660    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516663    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516665    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516668    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516671    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516673    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516675    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516678    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.518389 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516680    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516683    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516685    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516687    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.516690    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517463    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517471    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517474    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517476    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517479    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517482    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517485    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517487    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517490    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517493    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517501    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517504    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517507    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517510    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517516    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.518857 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517519    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517522    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517524    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517527    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517530    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517534    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517538    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517542    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517545    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517547    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517550    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517553    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517556    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517558    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517561    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517564    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517566    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517569    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517572    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.519353 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517574    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517577    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517579    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517582    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517585    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517587    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517590    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517592    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517595    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517603    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517606    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517609    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517613    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517615    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517618    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517621    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517623    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517626    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517629    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517632    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.519790 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517634    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517637    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517639    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517642    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517644    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517646    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517649    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517653    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517657    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517659    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517663    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517665    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517668    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517670    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517673    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517675    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517678    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517680    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517682    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.520306 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517685    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517687    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517690    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517698    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517702    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517706    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517708    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517711    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517713    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517716    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517719    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517721    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517724    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517726    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517730    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517735    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517739    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517743    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517747    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.520734 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517752    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517756    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517760    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517765    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517770    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517773    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517775    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517777    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517781    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517784    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517786    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517789    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517791    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.517794    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518348    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518354    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518363    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518366    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518373    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518376    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.521178 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518378    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518381    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518383    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518386    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518390    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518392    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518395    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518398    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518401    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518403    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518406    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518408    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518411    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518413    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518416    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518418    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518421    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518423    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518426    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518428    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.522767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518431    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518434    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518436    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518439    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518441    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518444    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518446    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518449    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518451    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518461    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518468    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518471    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518474    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518477    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518480    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518482    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518485    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518487    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518489    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.523595 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518492    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518494    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518496    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518499    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518502    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518504    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518506    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518509    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518511    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518513    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518516    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518518    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518521    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518523    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518526    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518528    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518531    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518534    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518536    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518538    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.524410 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518541    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518544    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518546    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518557    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518560    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518564    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518567    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518570    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518574    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518576    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518579    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518581    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518584    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518586    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518589    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518591    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518594    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518596    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518599    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518601    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.525245 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518603    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518606    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518608    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518611    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518613    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518616    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518618    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518621    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518623    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518625    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518628    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518630    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518633    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518635    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518638    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518642    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518650    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518661    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518664    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.526093 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518667    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.518669    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518761    5580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518773    5580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518780    5580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518784    5580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518788    5580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518791    5580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518796    5580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518800    5580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518803    5580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518806    5580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518809    5580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518812    5580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518815    5580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518819    5580 flags.go:64] FLAG: --cgroup-root=""
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518821    5580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518824    5580 flags.go:64] FLAG: --client-ca-file=""
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518827    5580 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518830    5580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518834    5580 flags.go:64] FLAG: --cluster-domain=""
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518836    5580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518839    5580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518842    5580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 21:35:01.526885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518845    5580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518849    5580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518852    5580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518855    5580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518860    5580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518863    5580 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518867    5580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518870    5580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518874    5580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518876    5580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518880    5580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518883    5580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518886    5580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518889    5580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518892    5580 flags.go:64] FLAG: --enable-server="true"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518895    5580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518899    5580 flags.go:64] FLAG: --event-burst="100"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518902    5580 flags.go:64] FLAG: --event-qps="50"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518905    5580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518909    5580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518912    5580 flags.go:64] FLAG: --eviction-hard=""
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518916    5580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518919    5580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518922    5580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518925    5580 flags.go:64] FLAG: --eviction-soft=""
Nov 02 21:35:01.527857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518928    5580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518930    5580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518933    5580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518936    5580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518939    5580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518942    5580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518945    5580 flags.go:64] FLAG: --feature-gates=""
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518949    5580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518952    5580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518971    5580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518976    5580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518983    5580 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518988    5580 flags.go:64] FLAG: --help="false"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518991    5580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518994    5580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.518998    5580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519001    5580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519004    5580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519007    5580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519010    5580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519013    5580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519016    5580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519018    5580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 21:35:01.528874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519021    5580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519024    5580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519027    5580 flags.go:64] FLAG: --kube-reserved=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519032    5580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519035    5580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519038    5580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519041    5580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519044    5580 flags.go:64] FLAG: --lock-file=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519046    5580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519049    5580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519052    5580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519057    5580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519060    5580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519063    5580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519066    5580 flags.go:64] FLAG: --logging-format="text"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519069    5580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519072    5580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519075    5580 flags.go:64] FLAG: --manifest-url=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519078    5580 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519082    5580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519087    5580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519090    5580 flags.go:64] FLAG: --max-pods="110"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519093    5580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519096    5580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519100    5580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 21:35:01.529814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519103    5580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519106    5580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519108    5580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519111    5580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519119    5580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519122    5580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519125    5580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519128    5580 flags.go:64] FLAG: --pod-cidr=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519131    5580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519134    5580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519139    5580 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519142    5580 flags.go:64] FLAG: --port="10250"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519145    5580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519148    5580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-010ad93c5847a9a8e"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519151    5580 flags.go:64] FLAG: --qos-reserved=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519154    5580 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519157    5580 flags.go:64] FLAG: --register-node="true"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519160    5580 flags.go:64] FLAG: --register-with-taints=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519164    5580 flags.go:64] FLAG: --registry-burst="10"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519167    5580 flags.go:64] FLAG: --registry-qps="5"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519170    5580 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519173    5580 flags.go:64] FLAG: --reserved-memory=""
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519177    5580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519180    5580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519183    5580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519185    5580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 21:35:01.530901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519188    5580 flags.go:64] FLAG: --runonce="false"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519193    5580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519196    5580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519199    5580 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519202    5580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519205    5580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519209    5580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519212    5580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519215    5580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519218    5580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519220    5580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519223    5580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519226    5580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519229    5580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519232    5580 flags.go:64] FLAG: --system-cgroups=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519235    5580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519242    5580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519244    5580 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519247    5580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519251    5580 flags.go:64] FLAG: --tls-min-version=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519254    5580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519256    5580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519259    5580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519262    5580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519265    5580 flags.go:64] FLAG: --v="2"
Nov 02 21:35:01.531795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519269    5580 flags.go:64] FLAG: --version="false"
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519273    5580 flags.go:64] FLAG: --vmodule=""
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519277    5580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519280    5580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519437    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519441    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519444    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519447    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519450    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519453    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519456    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519458    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519461    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519463    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519467    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519469    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519472    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519475    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519477    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519480    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.532743 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519482    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519485    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519488    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519492    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519494    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519497    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519499    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519502    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519505    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519507    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519510    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519512    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519515    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519518    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519520    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519523    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519525    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519528    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519530    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.533274 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519533    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519535    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519538    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519540    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519543    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519546    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519548    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519551    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519554    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519556    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519559    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519561    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519564    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519566    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519569    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519571    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519575    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519577    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519580    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519583    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.533811 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519585    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519588    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519590    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519594    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519597    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519600    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519603    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519606    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519608    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519611    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519613    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519616    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519619    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519621    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519623    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519626    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519629    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519631    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519634    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.534291 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519636    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519639    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519642    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519644    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519647    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519650    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519652    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519654    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519657    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519662    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519664    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519667    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519669    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519671    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519674    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519676    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519679    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519681    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519684    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519686    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.534767 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519689    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519692    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519695    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519699    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519703    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519705    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519708    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519710    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519713    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519715    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519717    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519720    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.519745    5580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519783    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519789    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.535541 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519793    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519797    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519800    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519803    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519806    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519808    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519811    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519815    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519817    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519819    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519822    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519825    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519827    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519832    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519834    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519837    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519839    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519842    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519845    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519848    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.536003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519850    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519853    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519855    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519858    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519860    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519863    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519865    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519868    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519871    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519873    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519876    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519878    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519881    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519884    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519887    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519889    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519892    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519894    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519897    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.536457 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519901    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519904    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519907    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519909    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519912    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519914    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519918    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519920    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519923    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519925    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519928    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519931    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519933    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519936    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519939    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519942    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519944    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519947    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519949    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519951    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.536910 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519968    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519972    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519976    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519981    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519985    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519989    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519991    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519995    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.519999    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520003    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520006    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520009    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520013    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520015    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520018    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520020    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520023    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520025    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520029    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.537419 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520032    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520034    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520037    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520039    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520042    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520044    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520047    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520049    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520052    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520054    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520057    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520060    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520062    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520065    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520067    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520070    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520072    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520075    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520077    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520080    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.537856 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520082    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520085    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520088    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520090    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520094    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.520097    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.524792    5580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.524809    5580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.524827    5580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.524835    5580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524914    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524921    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524926    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524931    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524935    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524940    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524944    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524948    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524968    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.538332 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524973    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524978    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524982    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524986    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524991    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524995    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.524999    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525004    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525008    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525012    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525017    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525021    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525025    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525030    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525036    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525040    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525045    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525050    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.538771 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525054    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525058    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525062    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525067    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525071    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525076    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525080    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525084    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525089    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525093    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525097    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525101    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525105    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525109    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525113    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525118    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525122    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525126    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525130    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525135    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.539216 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525139    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525143    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525147    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525151    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525155    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525160    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525163    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525167    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525172    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525182    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525186    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525190    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525195    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525198    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525203    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525207    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525212    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525216    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525220    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525224    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.539684 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525228    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525235    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525241    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525246    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525250    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525254    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525258    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525264    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525270    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525275    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525280    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525284    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525294    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525298    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525303    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525307    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525311    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525315    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525319    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.540235 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525324    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525329    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525334    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525339    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525344    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525348    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525352    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525356    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525360    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525365    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525369    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525373    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525377    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525381    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525385    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525389    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525393    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525397    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525401    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.540702 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525406    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.525446    5580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525672    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525680    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525685    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525689    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525693    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525697    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525702    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525706    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525711    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525715    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525719    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525723    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525730    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.541157 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525736    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525741    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525746    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525751    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525755    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525759    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525763    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525768    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525772    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525776    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525780    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525784    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525788    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525793    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525797    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525801    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525805    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525810    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525814    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.541508 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525818    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525823    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525827    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525830    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525835    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525840    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525844    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525848    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525852    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525856    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525861    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525865    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525869    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525873    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525878    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525881    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525886    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525890    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525894    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525898    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.541941 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525902    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525906    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525910    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525914    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525918    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525922    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525926    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525930    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525934    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525938    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525942    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525946    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525950    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525971    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525975    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525979    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525983    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525987    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525991    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525995    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.542416 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.525999    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526003    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526007    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526011    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526015    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526019    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526023    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526028    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526032    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526039    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526043    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526047    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526052    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526058    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526063    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526068    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526073    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526077    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526081    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.542905 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526085    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526089    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526093    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526097    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526101    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526105    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526109    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526113    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526117    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526121    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526126    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526129    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526134    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526138    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526142    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.526183    5580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 21:35:01.543346 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526233    5580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526240    5580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526245    5580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526249    5580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526254    5580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526258    5580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526262    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526268    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526272    5580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526276    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526281    5580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526285    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526289    5580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526293    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526297    5580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526301    5580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526305    5580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526309    5580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526313    5580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 21:35:01.543707 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526318    5580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526322    5580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526326    5580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526329    5580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526333    5580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526340    5580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526346    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526351    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526357    5580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526361    5580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526365    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526369    5580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526373    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526377    5580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526381    5580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526385    5580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526390    5580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526394    5580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526397    5580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 21:35:01.544159 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526402    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526406    5580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526411    5580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526415    5580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526419    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526423    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526427    5580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526431    5580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526435    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526439    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526444    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526449    5580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526453    5580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526457    5580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526461    5580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526465    5580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526469    5580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526473    5580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526477    5580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526481    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 21:35:01.544583 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526485    5580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526494    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526498    5580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526502    5580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526506    5580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526510    5580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526514    5580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526518    5580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526522    5580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526526    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526530    5580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526534    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526538    5580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526542    5580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526546    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526551    5580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526555    5580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526560    5580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526564    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526568    5580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 21:35:01.545121 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526572    5580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526576    5580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526579    5580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526584    5580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526587    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526591    5580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526595    5580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526599    5580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526603    5580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526607    5580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526611    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526616    5580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526620    5580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526626    5580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526630    5580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526634    5580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526638    5580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526642    5580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526646    5580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 21:35:01.545584 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526661    5580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526667    5580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526672    5580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526676    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526680    5580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526684    5580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526688    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526692    5580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:01.526696    5580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.526764    5580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528207    5580 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528265    5580 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528506    5580 server.go:1014] "Starting client certificate rotation"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528644    5580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528694    5580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:28:49 +0000 UTC" deadline="2028-03-28 08:46:21.479121719 +0000 UTC"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.528719    5580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12275h11m19.950404024s"
Nov 02 21:35:01.546063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.529192    5580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:35:01.546646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.530744    5580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 21:35:01.546646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.532991    5580 log.go:25] "Validated CRI v1 runtime API"
Nov 02 21:35:01.546646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.534892    5580 log.go:25] "Validated CRI v1 image API"
Nov 02 21:35:01.546646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.535684    5580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 21:35:01.546646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.537610    5580 fs.go:135] Filesystem UUIDs: map[1485b32b-8ea0-4aff-96fc-2e645beb302a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b45cc05b-d07a-41ff-80d1-7c53c2bec687:/dev/nvme0n1p3]
Nov 02 21:35:01.546922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.537631    5580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76/userdata/shm major:0 minor:307 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb/userdata/shm major:0 minor:379 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3fe9687f35d72796591c0499551d12a4244a2030b407969df266d8b423f83e44/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3fe9687f35d72796591c0499551d12a4244a2030b407969df266d8b423f83e44/userdata/shm major:0 minor:258 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/464a3c3d5b4c39b207b152bffd919fc9c2acf949cd8d5993d86d43f135c41391/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/464a3c3d5b4c39b207b152bffd919fc9c2acf949cd8d5993d86d43f135c41391/userdata/shm major:0 minor:256 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85/userdata/shm major:0 minor:414 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437/userdata/shm major:0 minor:326 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6/userdata/shm major:0 minor:419 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef/userdata/shm major:0 minor:421 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/tmp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~projected/kube-api-access-hhlwx:{mountpoint:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~projected/kube-api-access-hhlwx major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~projected/kube-api-access-c4268:{mountpoint:/var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~projected/kube-api-access-c4268 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9/volumes/kubernetes.io~projected/kube-api-access-7kr9g:{mountpoint:/var/lib/kubelet/pods/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9/volumes/kubernetes.io~projected/kube-api-access-7kr9g major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e177cbc-5f97-45c8-9714-73510ec5597b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/2e177cbc-5f97-45c8-9714-73510ec5597b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8/volumes/kubernetes.io~projected/kube-api-access-zj598:{mountpoint:/var/lib/kubelet/pods/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8/volumes/kubernetes.io~projected/kube-api-access-zj598 major:0 minor:413 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~projected/kube-api-access-mjrhh:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~projected/kube-api-access-mjrhh major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~projected/kube-api-access-qkzxs:{mountpoint:/var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~projected/kube-api-access-qkzxs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~secret/metrics-certs major:0 minor:390 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~projected/kube-api-access-c9gfl:{mountpoint:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~projected/kube-api-access-c9gfl major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~projected/kube-api-access-nlxgq:{mountpoint:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~projected/kube-api-access-nlxgq major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4905999a-0718-4c3d-8cfd-abe292d5e01c/volumes/kubernetes.io~projected/kube-api-access-fl9zx:{mountpoint:/var/lib/kubelet/pods/4905999a-0718-4c3d-8cfd-abe292d5e01c/volumes/kubernetes.io~projected/kube-api-access-fl9zx major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~projected/kube-api-access-shdfv:{mountpoint:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~projected/kube-api-access-shdfv major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c/volumes/kubernetes.io~projected/kube-api-access-vdsrg:{mountpoint:/var/lib/kubelet/pods/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c/volumes/kubernetes.io~projected/kube-api-access-vdsrg major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~projected/kube-api-access-59sw4:{mountpoint:/var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~projected/kube-api-access-59sw4 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/config-reloader/2 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/prometheus/15 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~empty-dir/config-out major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/kube-api-access-fmd99:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/kube-api-access-fmd99 major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/tls-assets major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/config major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/web-config major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/942b2063-cdbd-49a9-bb85-21053e8f15c6/volumes/kubernetes.io~projected/kube-api-access-gbkqw:{mountpoint:/var/lib/kubelet/pods/942b2063-cdbd-49a9-bb85-21053e8f15c6/volumes/kubernetes.io~projected/kube-api-access-gbkqw major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~projected/kube-api-access-p96zw:{mountpoint:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~projected/kube-api-access-p96zw major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab35d4ec-3432-43b7-adca-34097c507ae1/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ab35d4ec-3432-43b7-adca-34097c507ae1/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~projected/kube-api-access-mvs7m:{mountpoint:/var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~projected/kube-api-access-mvs7m major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b35ac1d5-682e-4eab-9a5b-a0675b542860/volumes/kubernetes.io~projected/kube-api-access-ns6bz:{mountpoint:/var/lib/kubelet/pods/b35ac1d5-682e-4eab-9a5b-a0675b542860/volumes/kubernetes.io~projected/kube-api-access-ns6bz major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~projected/kube-api-access-j452n:{mountpoint:/var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~projected/kube-api-access-j452n major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e88de0d5-7833-4038-8119-201c5fff01f5/volumes/kubernetes.io~projected/kube-api-access-597wm:{mountpoint:/var/lib/kubelet/pods/e88de0d5-7833-4038-8119-201c5fff01f5/volumes/kubernetes.io~projected/kube-api-access-597wm major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/afa4464e7cea12757570f913b58b339b501c88e67bc8c6832c5b8217b30c7535/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/194e08a9f3c89ff9eaadb3b1c0acd2eab08cde9867d3cd43146b8501fdcfd049/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/cff3889022c463ed480f23cab4a9b89c7b0b3a3a2d1eb7cde2c039df8bedcdf4/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/5bfdc7ed1b7113ad23f8ef870d585bdd2d5d42369d1c863373812b626102de6e/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/6c625518000cbb8cea194a1eb2553b33a72100c50ea2b345683a01bdfe8e1753/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/a059de8d9f20e56f6d946f7ce1efc853c62b1be909e6684ce49275de76ca17bb/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/81612ab229b65a8498cd7ded7f44a2a3222cb9446adbcb17428bf14b0d1b03d4/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/bb478015233da68c219fda6cc07309c976980a79be206952c1051fc7a6285096/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/77f1fc8e619cba6b968a75d88e428f7d7868852537d16056972c11fe507637ed/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/89e0da7269504a6691de8278e922644696c7ef9528f4d30d76df6b9ed58c000b/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/13879c5a671c01a91e0a8ebeb30fddf7f66c52291e35e19c3dc02e46a4fc2416/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/4a0046b2a6d43f779bab27b3da9af01c1358a432e0c13814e02c2a6a90dd580d/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/7bc3526bcda539c653f5b860176837b7fbb15e5ba2aa6e74a68213beddff8c45/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/bdadba2f1b586b2bc003afc2452e7c83d116b14f2e1566875aba69bd9a0b5c84/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/414d0f374426c7af9aae5b2f015ab0d91b984f09df78414e9633ff3d0eca55ce/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/cde52ab4dcfc1d5778028796af702f254e1fa3f2caa07d6ab059106fcf4efc10/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/8d2230d8d974c75782af3c61cdbb567f479eca1a1b752a9fc85a818886335086/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/14e34fa0b81bda44a5cb139e9c74e1d80342fb05e69800460ec2201d5762eda7/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/993ddb3285abea284183a5aaf91e2ee9092ec566af4f5fafa1eeb5444c7ab8f7/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/ad4178120c870522a913ea2fd61cbab656b07a78629b5e813e2ddb514a6d5ed1/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/75594a7eee857111f6d009e5fec534e2c2aaeeba89dc050ab4d2a65118a30744/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/ff7e0dea2ad480f5dc6ffa3099de6beff4bbe9a2c2012ef80a938a19757d64c7/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/03750e0337764378b1d3bea5dbaf1d4fa44bb1067672ad416e2ecc3f8be4cdb7/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/bc83a8e901826f13dd5c4d06bf5580b9469b05ce9c4e6f19337d2df95e613d90/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/8efdc442335b4145f96ca57cd1590a360dbbe121ce708e6a774638980acc5baf/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/ffeb66edbc7dd6565a04339c47ff57cee1b641972e7b5fb1239a8405043c2fba/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/a13e96f5a7933b72d47acf81412f4027d0ce0626658491c8e48e8039fc3943d5/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/400d67c702d1ee6fa7f6c88f1ef870810e04e6209f6c4b8d0f3191eab43d05b6/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/763bf588c920beee685c9a62c8327948e3788a45497de5d15e74cbcea0529e13/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/bda57404709434d2910cefa448d18e18b820b1e19fe1b15b7f23255840d11e41/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/19b4e1066c3775dd20e4198c42b145502cca90138f965b4ab72d64d110b5b544/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/6f2cc0873ba1f5992613d92a566d1a94f71f4246c3bf8e7776c2536e0086012f/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/9d5d1a8dd2a35736d605bc4723724c526470ae88c9f341151eec82a9f9d7eac6/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/0721dec58143ef98773805e6573e2ddc7c78782d533d852306c93f8e31ee0f89/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/e706e63112679d804e54d73972edceb9db25b384cb86eff018dccae73ca264bd/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/00fcc60df896d905f1b4c9b763ca6fb8a05660338a42a263646b34b04f5dbd0d/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/39bc20adcde2f7c80c97d3c0c336a9323662669c4a9cd2299395b50d66cd18ad/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/9e8f576296d98b07839a72e81d2e61b9f5b7e6b3414ba1ee5a10357f242b53a8/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/36618b995f24cb537329e74e1b436c06de4830e91ab9d034ad1542fd48e9234e/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/b41d513beed8469d6ba46533ee4233e1061c25268aaf4691b0415d1d46ab9b40/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/43aa430ff93a46d429249ab2c2f52cd2fb50f9f75db1a3822e003155e8d1f7b3/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/57373823f4c17a735487ab36bf8c3c851dcb0292bd71e48be08e93945c3f014c/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/8acb6a9f6d245fcefb10d16a5d2a5e27685e9f2c991659ac4cd7e417f8a54f3e/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/d466ddde9b1d42eb44b09a1e66e2b5f90b3d62fd3046837a54dee1aa15295b9c/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/870056dec6f82411943d884176f69eb74ebb4f410c4755ca3a7e1e33e065bdf0/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/9c42254dfbef8d339d52b349c67fd5c83c11ffd363ab659d4b1db00eea3cc8e4/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/6ea6fa767331ef55645440a62a4489b908d455671405953c1a6087e67df9611f/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/a936a87e50b911abcf3f3b6cf4b0ba5aab6c47eaa5242b636a16ec7e09a868b5/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/dd907b170f6932bc130037c6061d9b2ccfe90de43c5446b73f6d1eb25e097870/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/0882bbc59e32912e556ef6e8b35352669d32399b6216b90303998502fbb0417e/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/7e61fda1aeae99356df0a14067cd9ad1627044d3111236da3e4ffb287901275c/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-383:{mountpoint:/var/lib/containers/storage/overlay/79e774bce65840630fda5a7404be2143f2f9cec1d359b172f07eb101554a5fe1/merged major:0 minor:383 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/bc0f7611e5cc2b8058ed8765fc3049bc6d0fdb36fa87259f61cf671ef2dea136/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/707dadf3b66b833b097ba56a025ad88e27dd2d1f758aa05b5b515a4416f8a922/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/0717b8557457dddec803fa0bdfb410675f931d92d4809a3822fcb760331648e6/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/fc45ea0036176791acb478b35d339a756e705e5fa6ef56264b68b53e7f863379/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/f531d225f75943debabb1f68797ae24a5a7bc5e213ec66b43a99cb3a88b8f47e/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/7904945416b5dc30af2a26170fff6c36adc9ba8c72078ba6a0b9fbd326ea60d6/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/754ea7f1cbfc5388f626296e8f850a99275defc3b0dd5a5ba8517d00a67379b8/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/12f6ed01639cf92063774d6a59ad22704a40b13c717978e825bf6a130973b7cf/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/ac08b0432f1ba803a78795c6deada30653f18a6b93bb06f3112686768010c6ef/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/6b74d9d4be373e76709ada0816f42e28a5ecb44d36b58f109a2ea6005f8fb4fe/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/0d39dae364620854a60ecba925d7cf2ae70a6bb037a37308bcbce15301e877d4/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/496a7346f998d2f6ec2c38807c0f15bd81a18d7f7f780411707604b0398944e9/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/8b0380c715bf5bef74f8641e0223895bdd9cb06dca2e4e748b252cfc01dafc7e/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/4fcfb9953372a3674188776147f960c9b6da3981352ca7d7a6ee3a28a5ec66c0/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/5b46d75334ada31f2dc7ff196fb4ebfa5830aebf61c353436d810a87ce7c3f7b/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/73888b05d2326ef695532dbd3e8ade5f01a2dd8b8d899312ad7f45be075a1c8c/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/8365d2b2373ca205025831776f6ff06ceb2afecda16666aa139d319034f73edc/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/94594a121b35c5833a290f13d673035c0877ff3abe45c0a14c8f9fc750df865d/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/e5069e0642a40fb34bc15b091ead5c69aee6269355f817ee2c0c8f6437bd0445/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/4f2cd1eec967468c9773ba236262254a65cad1f02d45424ba0715e4ab778bde5/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ca7f71e1bd066cf739cd56f2d34368e510b16f5ed41638504504b794d21e94bb/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/d41231c8467d63ffa906af5c883b7ae4abf7f66907e033da98af236ee0165088/merged major:0 minor:97 fsType:overlay blockSize:0}]
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.544612    5580 manager.go:217] Machine: {Timestamp:2026-11-02 21:35:01.544164398 +0000 UTC m=+0.060660162 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203645 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd87621b33c7e81b7e0b3dbf03915 SystemUUID:ec2cd876-21b3-3c7e-81b7-e0b3dbf03915 BootID:7cabbb07-2d5e-404b-a262-bca470193391 Filesystems:[{Device:/var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:377 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce9b36d2664517da054bc4f616ddd1f1dd1f6df324106397f93d1347df2ce11b/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:375 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c/volumes/kubernetes.io~projected/kube-api-access-vdsrg DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1c84bd06688c992059ac377c3fe98aac3acb7539d8e5fcd82fcd912ddef8818/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:233 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:368 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b35ac1d5-682e-4eab-9a5b-a0675b542860/volumes/kubernetes.io~projected/kube-api-access-ns6bz DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-383 DeviceMajor:0 DeviceMinor:383 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:369 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:368 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~projected/kube-api-access-shdfv DeviceMajor:0 DeviceMinor:237 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:287 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e88de0d5-7833-4038-8119-201c5fff01f5/volumes/kubernetes.io~projected/kube-api-access-597wm DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88b8de092ace456c26803798191eb827a9bf23145b4660f470b3091c4fea46c5/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cf5a3c0a8e002a45aef040b018bed23fbe1476db5a55d46f50c2a4559e57a1f/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b99d94c9fff2fd1032d0b3473cb4614b7b0d86186e518d40821b36a913b2b8b6/userdata/shm DeviceMajor:0 DeviceMinor:419 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5a6f0ea3dfca2b9debe81477b74984b1fc66a2c595b6e2074ea5f1bb34763bb8/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:374 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~projected/kube-api-access-nlxgq DeviceMajor:0 DeviceMinor:238 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb/userdata/shm DeviceMajor:0 DeviceMinor:379 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8361144a2b748395583e67e236c670bbea2cb384530bcfd24f80d8c6db15177e/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4e1a1df8-8a6a-4136-9444-9eb795619aed/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:235 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aec2dacc-d493-48f3-9bec-45e232e24e4c/volumes/kubernetes.io~projected/kube-api-access-mvs7m DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d9e9164648006cb56b3da12778bbaaf147556597023eaf1921b41fdff938f23c/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:364 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/080207d050868ac67cf5ba6f22917c29f0dcbdd0905cd518e1e33843d44995b7/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6ca8e2a90fafaeaf676ada875396a7d4045b58318e5a6daccbc43cb53f5227eb/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:324 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~projected/kube-api-access-fmd99 DeviceMajor:0 DeviceMinor:378 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:370 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:229 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/103a29ef-eca9-4925-93df-023ee970d7f7/volumes/kubernetes.io~projected/kube-api-access-hhlwx DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:376 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~projected/kube-api-access-qkzxs DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~projected/kube-api-access-c9gfl DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e732cfa97a30f7b473945558c76dba9a7874608c30e80e00939200de88627fb9/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/942b2063-cdbd-49a9-bb85-21053e8f15c6/volumes/kubernetes.io~projected/kube-api-access-gbkqw DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9/volumes/kubernetes.io~projected/kube-api-access-7kr9g DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9b5a8906bf6ec2ab069ca80adffb4352ddd9e5a13559b477d4dae2ab017e9ef/userdata/shm DeviceMajor:0 DeviceMinor:421 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:286 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3c18553f-6a3e-4c2a-a4e1-424c05f27216/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:390 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8/volumes/kubernetes.io~projected/kube-api-access-zj598 DeviceMajor:0 DeviceMinor:413 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:234 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/459afbc7-52e2-42f9-98f0-b157dfc74671/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:236 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/464a3c3d5b4c39b207b152bffd919fc9c2acf949cd8d5993d86d43f135c41391/userdata/shm DeviceMajor:0 DeviceMinor:256 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:289 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:372 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/70ca2a762ca56ae8a13627da3c3fb5b12ae36a547799e8f8536b050937af9787/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:291 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e58fab09-a42f-4f63-96a9-cfe58f7ed15c/volumes/kubernetes.io~projected/kube-api-access-j452n DeviceMajor:0 DeviceMinor:325 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:368 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b3fb92a-6bfe-4d2f-968e-81741b324547/volumes/kubernetes.io~projected/kube-api-access-c4268 DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~projected/kube-api-access-p96zw DeviceMajor:0 DeviceMinor:230 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3fe9687f35d72796591c0499551d12a4244a2030b407969df266d8b423f83e44/userdata/shm DeviceMajor:0 DeviceMinor:258 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:371 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7d83a7a1-7653-4b92-ada5-137be7518e4e/volumes/kubernetes.io~projected/kube-api-access-59sw4 DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:368 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3f84beb-14a1-406c-924e-3896b29f6564/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~projected/kube-api-access-mjrhh DeviceMajor:0 DeviceMinor:293 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/339585b5684e88a63e5038bcaebb6f922c378f2cf12da9110269b58f4712ea76/userdata/shm DeviceMajor:0 DeviceMinor:307 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ef7602d3006a16a082674ab79f8ae81ceaa70c5759d60e73870d87961381f85/userdata/shm DeviceMajor:0 DeviceMinor:414 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b4488df9d305a0cbf5917d1e6cea38787cb3948d5ee9f2f86d8da98e520e6437/userdata/shm DeviceMajor:0 DeviceMinor:326 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2e177cbc-5f97-45c8-9714-73510ec5597b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:385 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ab35d4ec-3432-43b7-adca-34097c507ae1/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3be53397-eac6-47b3-8b63-faa5479e1f3b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:292 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/4905999a-0718-4c3d-8cfd-abe292d5e01c/volumes/kubernetes.io~projected/kube-api-access-fl9zx DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ddcd7cf250cb9bed40262ba2236dbd5acf6e9388fcbb2d89d4691974ea4caa0f/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:373 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:339585b5684e88a MacAddress:da:c1:3c:c8:ac:08 Speed:10000 Mtu:8901} {Name:36bf49f9220445c MacAddress:3e:4f:39:64:96:cc Speed:10000 Mtu:8901} {Name:464a3c3d5b4c39b MacAddress:ce:a5:3b:ea:b9:ef Speed:10000 Mtu:8901} {Name:5ef7602d3006a16 MacAddress:ee:ab:15:7c:eb:10 Speed:10000 Mtu:8901} {Name:6ca8e2a90fafaea MacAddress:a6:aa:e3:8f:6a:c7 Speed:10000 Mtu:8901} {Name:70ca2a762ca56ae MacAddress:5e:cb:5c:b9:52:5b Speed:10000 Mtu:8901} {Name:b4488df9d305a0c MacAddress:9e:cd:c3:47:ad:95 Speed:10000 Mtu:8901} {Name:b99d94c9fff2fd1 MacAddress:16:7e:de:89:da:20 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:69:ac:39:22:37 Speed:0 Mtu:9001} {Name:br-int MacAddress:ca:43:e6:a9:e7:13 Speed:0 Mtu:8901} {Name:ce9b36d2664517d MacAddress:ea:0a:bc:9d:89:70 Speed:10000 Mtu:8901} {Name:d9e9164648006cb MacAddress:3a:4e:1f:f5:1e:3a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:69:ac:39:22:37 Speed:0 Mtu:9001} {Name:f9b5a8906bf6ec2 MacAddress:72:20:c2:dc:b9:e3 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:b2:1f:cf:7f:45:e5 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:16:46:94:ec:71:4c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.544990    5580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545028    5580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545322    5580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545345    5580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-74.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545473    5580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545480    5580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545494    5580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545507    5580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545589    5580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545751    5580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545871    5580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545907    5580 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545919    5580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545928    5580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545936    5580 kubelet.go:405] "Adding apiserver pod source"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.545944    5580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.546551    5580 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.546560    5580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:35:01.547217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.547059    5580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 21:35:01.550815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.550796    5580 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 21:35:01.550815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.550813    5580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 21:35:01.550923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.550904    5580 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 21:35:01.551100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551089    5580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 21:35:01.551132 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551117    5580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 21:35:01.551209 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551201    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 21:35:01.551237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551215    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 21:35:01.551237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551221    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 21:35:01.551237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551228    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 21:35:01.551237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551233    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 21:35:01.551237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551238    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551244    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551250    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551256    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551262    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551270    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551278    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551304    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 21:35:01.551363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.551309    5580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 21:35:01.554417 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.554404    5580 server.go:1263] "Started kubelet"
Nov 02 21:35:01.554548 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.554495    5580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 21:35:01.554548 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.554503    5580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 21:35:01.554643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.554582    5580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 21:35:01.554907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.554891    5580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 21:35:01.554932 ip-10-0-136-74 systemd[1]: Started Kubernetes Kubelet.
Nov 02 21:35:01.555774 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.555759    5580 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 21:35:01.557784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557770    5580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557800    5580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557827    5580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557845    5580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557909    5580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:28:51 +0000 UTC" deadline="2028-06-20 04:32:54.459674789 +0000 UTC"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557970    5580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14286h57m52.901722687s"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.557945    5580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.558033    5580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.558776    5580 factory.go:55] Registering systemd factory
Nov 02 21:35:01.559097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.558791    5580 factory.go:223] Registration of the systemd container factory successfully
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559117    5580 factory.go:153] Registering CRI-O factory
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559133    5580 factory.go:223] Registration of the crio container factory successfully
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559216    5580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559235    5580 factory.go:103] Registering Raw factory
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559253    5580 manager.go:1199] Started watching for new ooms in manager
Nov 02 21:35:01.559912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.559691    5580 manager.go:319] Starting recovery of all containers
Nov 02 21:35:01.561911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.561871    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Nov 02 21:35:01.562010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.561924    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Nov 02 21:35:01.562010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.561939    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab35d4ec-3432-43b7-adca-34097c507ae1" volumeName="kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca" seLinuxMountContext=""
Nov 02 21:35:01.562010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.561980    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aec2dacc-d493-48f3-9bec-45e232e24e4c" volumeName="kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert" seLinuxMountContext=""
Nov 02 21:35:01.562010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.561994    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b35ac1d5-682e-4eab-9a5b-a0675b542860" volumeName="kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script" seLinuxMountContext=""
Nov 02 21:35:01.562205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.562007    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 02 21:35:01.562205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.562039    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b3fb92a-6bfe-4d2f-968e-81741b324547" volumeName="kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume" seLinuxMountContext=""
Nov 02 21:35:01.562205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.562052    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b3fb92a-6bfe-4d2f-968e-81741b324547" volumeName="kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir" seLinuxMountContext=""
Nov 02 21:35:01.562205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.562074    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c18553f-6a3e-4c2a-a4e1-424c05f27216" volumeName="kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs" seLinuxMountContext=""
Nov 02 21:35:01.562205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.562088    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 02 21:35:01.563681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.563648    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy" seLinuxMountContext=""
Nov 02 21:35:01.567283 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567183    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b3fb92a-6bfe-4d2f-968e-81741b324547" volumeName="kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268" seLinuxMountContext=""
Nov 02 21:35:01.567372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567286    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="942b2063-cdbd-49a9-bb85-21053e8f15c6" volumeName="kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config" seLinuxMountContext=""
Nov 02 21:35:01.567372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567301    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f84beb-14a1-406c-924e-3896b29f6564" volumeName="kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 02 21:35:01.567372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567313    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab35d4ec-3432-43b7-adca-34097c507ae1" volumeName="kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs" seLinuxMountContext=""
Nov 02 21:35:01.567372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567347    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="103a29ef-eca9-4925-93df-023ee970d7f7" volumeName="kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned" seLinuxMountContext=""
Nov 02 21:35:01.567571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567364    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config" seLinuxMountContext=""
Nov 02 21:35:01.567571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567419    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file" seLinuxMountContext=""
Nov 02 21:35:01.567571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.567435    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f84beb-14a1-406c-924e-3896b29f6564" volumeName="kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca" seLinuxMountContext=""
Nov 02 21:35:01.570228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570157    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e88de0d5-7833-4038-8119-201c5fff01f5" volumeName="kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm" seLinuxMountContext=""
Nov 02 21:35:01.570312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570246    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Nov 02 21:35:01.570312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570263    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Nov 02 21:35:01.570312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570283    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="103a29ef-eca9-4925-93df-023ee970d7f7" volumeName="kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp" seLinuxMountContext=""
Nov 02 21:35:01.570312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570297    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 02 21:35:01.570502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570329    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 02 21:35:01.570502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570384    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs" seLinuxMountContext=""
Nov 02 21:35:01.570502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570430    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aec2dacc-d493-48f3-9bec-45e232e24e4c" volumeName="kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m" seLinuxMountContext=""
Nov 02 21:35:01.570502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570451    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570514    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570530    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570584    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570602    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" volumeName="kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570635    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570663    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca" seLinuxMountContext=""
Nov 02 21:35:01.570683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570677    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570695    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f254a4a-36cb-4152-8c7e-b9a1c441c6d8" volumeName="kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570745    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570757    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570792    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f84beb-14a1-406c-924e-3896b29f6564" volumeName="kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570811    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" volumeName="kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570821    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570828    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c18553f-6a3e-4c2a-a4e1-424c05f27216" volumeName="kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570842    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570854    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4905999a-0718-4c3d-8cfd-abe292d5e01c" volumeName="kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570882    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570892    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e88de0d5-7833-4038-8119-201c5fff01f5" volumeName="kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570900    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d83a7a1-7653-4b92-ada5-137be7518e4e" volumeName="kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570909    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570939    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b3fb92a-6bfe-4d2f-968e-81741b324547" volumeName="kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls" seLinuxMountContext=""
Nov 02 21:35:01.571001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.570947    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571027    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4905999a-0718-4c3d-8cfd-abe292d5e01c" volumeName="kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571037    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571091    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="103a29ef-eca9-4925-93df-023ee970d7f7" volumeName="kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571102    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" volumeName="kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571118    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b35ac1d5-682e-4eab-9a5b-a0675b542860" volumeName="kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571159    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e58fab09-a42f-4f63-96a9-cfe58f7ed15c" volumeName="kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571217    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e58fab09-a42f-4f63-96a9-cfe58f7ed15c" volumeName="kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571226    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e88de0d5-7833-4038-8119-201c5fff01f5" volumeName="kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571253    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e88de0d5-7833-4038-8119-201c5fff01f5" volumeName="kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571296    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" volumeName="kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571339    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571353    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571371    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571385    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d83a7a1-7653-4b92-ada5-137be7518e4e" volumeName="kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571392    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571421    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571455    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9" volumeName="kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571470    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e177cbc-5f97-45c8-9714-73510ec5597b" volumeName="kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571526    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571539    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3be53397-eac6-47b3-8b63-faa5479e1f3b" volumeName="kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571554    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571565    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1a1df8-8a6a-4136-9444-9eb795619aed" volumeName="kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571592    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out" seLinuxMountContext=""
Nov 02 21:35:01.571627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571611    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571632    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c" volumeName="kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571678    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c" volumeName="kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571702    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d83a7a1-7653-4b92-ada5-137be7518e4e" volumeName="kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571716    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571741    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="880e1a9f-5711-436e-938f-619f3e692a56" volumeName="kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571756    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="942b2063-cdbd-49a9-bb85-21053e8f15c6" volumeName="kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571808    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="942b2063-cdbd-49a9-bb85-21053e8f15c6" volumeName="kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571823    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3f84beb-14a1-406c-924e-3896b29f6564" volumeName="kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571846    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" volumeName="kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571877    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="459afbc7-52e2-42f9-98f0-b157dfc74671" volumeName="kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571936    5580 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d83a7a1-7653-4b92-ada5-137be7518e4e" volumeName="kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.571994    5580 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 21:35:01.572642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.572007    5580 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 21:35:01.584040 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.584007    5580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 21:35:01.585626 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.585604    5580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 21:35:01.585626 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.585628    5580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 21:35:01.585784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.585653    5580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 21:35:01.585784 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.585698    5580 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 21:35:01.598423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.598398    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:35:01.600082 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.600059    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:35:01.605853 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.605838    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:35:01.606204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.606178    5580 generic.go:356] "Generic (PLEG): container finished" podID="3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7" containerID="e15f7b47f687f48611670da126da4aa208d87cdaf9dcaf0552f01a971e25c92f" exitCode=1
Nov 02 21:35:01.609811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.609781    5580 generic.go:356] "Generic (PLEG): container finished" podID="be4a2511be029007da6e29235975f6dc" containerID="2ca236d176a2f93e30058df56a4f79b2ce18a552a651a3f3b284b12934e52060" exitCode=0
Nov 02 21:35:01.623082 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623060    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="49a748c1d4fa17981fd20577db47e8547d5862e394a2dc4c902dcc2d8750422f" exitCode=0
Nov 02 21:35:01.623082 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623081    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="2f3e2cb54fbbd371854f27e1051579db33d52ee320e9dac1308b53af859139b6" exitCode=0
Nov 02 21:35:01.623220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623090    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="a86e0011a02fc4382475f413ac52099d04b82e9cbc6ec2511356b506d463b693" exitCode=0
Nov 02 21:35:01.623220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623095    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="60de31f65695e19341d357aab3f320c7bc9ccc2d20996dbd571e23c501456b93" exitCode=0
Nov 02 21:35:01.623220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623100    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="d6b572e9558e6841ac16f6b625db533874763657f24faec651ed909c37634ad0" exitCode=0
Nov 02 21:35:01.623220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.623105    5580 generic.go:356] "Generic (PLEG): container finished" podID="e88de0d5-7833-4038-8119-201c5fff01f5" containerID="1bc73d2b91bdd1ca39cbbc99659d3a36f18055aab7fba336c787feeb5169aac0" exitCode=0
Nov 02 21:35:01.630303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.630278    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b" exitCode=0
Nov 02 21:35:01.633923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.633899    5580 generic.go:356] "Generic (PLEG): container finished" podID="459afbc7-52e2-42f9-98f0-b157dfc74671" containerID="603ff591fe76e02e5e0e906cbb9c4c32773c8df7ee8fe271fa7ae31783df8f0c" exitCode=0
Nov 02 21:35:01.658118 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.658097    5580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:35:01.670720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.670702    5580 manager.go:324] Recovery completed
Nov 02 21:35:01.686600 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.686580    5580 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 02 21:35:01.688854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.688841    5580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:35:01.690786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.690761    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:35:01.690856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.690804    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:35:01.690856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.690821    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:35:01.694150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694092    5580 cpu_manager.go:226] "Starting" policy="none"
Nov 02 21:35:01.694150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694113    5580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 21:35:01.694150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694133    5580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 21:35:01.694339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694322    5580 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 21:35:01.694386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694342    5580 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 21:35:01.694386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694367    5580 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 21:35:01.694386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694376    5580 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 21:35:01.694386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694383    5580 policy_none.go:50] "Start"
Nov 02 21:35:01.694547 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694406    5580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 21:35:01.694547 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694419    5580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 21:35:01.694668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694647    5580 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 21:35:01.694717 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694671    5580 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 21:35:01.694717 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.694680    5580 policy_none.go:44] "Start"
Nov 02 21:35:01.695437 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.695413    5580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:35:01.696509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.696487    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:35:01.696596 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.696525    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:35:01.696596 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.696537    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:35:01.699779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.699753    5580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 21:35:01.702412 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702399    5580 manager.go:341] "Starting Device Plugin manager"
Nov 02 21:35:01.702474 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.702424    5580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 21:35:01.702474 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702433    5580 server.go:87] "Starting device plugin registration server"
Nov 02 21:35:01.702756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702743    5580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 21:35:01.702787 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702760    5580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 21:35:01.702854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702831    5580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 21:35:01.703013 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.702973    5580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 21:35:01.703013 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.703016    5580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 21:35:01.703171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.703023    5580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 21:35:01.703171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.703066    5580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T21:35:01.703006666Z","UUID":"2a151d44-21ca-49d2-86f5-7434c83f6094","Handler":null,"Name":"","Endpoint":""}
Nov 02 21:35:01.704402 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.704385    5580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 21:35:01.704478 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.704420    5580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 21:35:01.704736 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.704714    5580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 21:35:01.704797 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.704754    5580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-74.us-west-1.compute.internal\" not found"
Nov 02 21:35:01.756949 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.756924    5580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:35:01.756949 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.756941    5580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:35:01.757119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.757071    5580 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 21:35:01.757933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.757915    5580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:35:01.803878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.803856    5580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 21:35:01.804699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.804686    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 21:35:01.804755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.804712    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 21:35:01.804755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.804722    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 21:35:01.804831 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.804759    5580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.855258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.855234    5580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 21:35:01.887165 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887118    5580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"]
Nov 02 21:35:01.887670 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887635    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"8ed0968c21495795a0292b52f44b9b55","Type":"ContainerStarted","Data":"9c60796e3404840c6286b30b3a377b62d11c14488f93aad116dd10e6c9562be8"}
Nov 02 21:35:01.887713 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887681    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"8ed0968c21495795a0292b52f44b9b55","Type":"ContainerStarted","Data":"a2a414b0fa2f077b373dbc84f2daf2c8903a0f3f6ed2de0790a4448332a2c6e2"}
Nov 02 21:35:01.887749 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887735    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerStarted","Data":"0028982bd254904f355007224b3dea79916abb577735a30022f129d384ce969a"}
Nov 02 21:35:01.887791 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887760    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerDied","Data":"2ca236d176a2f93e30058df56a4f79b2ce18a552a651a3f3b284b12934e52060"}
Nov 02 21:35:01.887791 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887780    5580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.887852 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887824    5580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.887882 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.887781    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal" event={"ID":"be4a2511be029007da6e29235975f6dc","Type":"ContainerStarted","Data":"742e11675d9725e5687c7ae691756d6e2db26d8c05f1a4e4fdbc494ca9cd7b43"}
Nov 02 21:35:01.911993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.911974    5580 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.912056 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.912021    5580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.912430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.912417    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeNotReady"
Nov 02 21:35:01.912468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.912442    5580 setters.go:546] "Node became not ready" node="ip-10-0-136-74.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T21:35:01Z","lastTransitionTime":"2026-11-02T21:35:01Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 21:35:01.948320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.948255    5580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:35:01.948320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.948255    5580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:35:01.948320 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.948312    5580 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.948488 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:01.948328    5580 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:01.967619 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:01.967595    5580 nodeinfomanager.go:471] existing CSINode "ip-10-0-136-74.us-west-1.compute.internal" is owned by different node (oldNodeID="5fbbdf86-a366-44df-ad3a-774b3f598324", newNodeID=""), cleaning up...
Nov 02 21:35:02.057983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.057944    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.058086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.057989    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.058086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.058023    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158190    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158220    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158236    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158257    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158277    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ed0968c21495795a0292b52f44b9b55-config\") pod \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"8ed0968c21495795a0292b52f44b9b55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.158368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.158298    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4a2511be029007da6e29235975f6dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" (UID: \"be4a2511be029007da6e29235975f6dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.546271 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.546246    5580 apiserver.go:52] "Watching apiserver"
Nov 02 21:35:02.650601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.650572    5580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.651211 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.651193    5580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.651812 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.651798    5580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 21:35:02.652332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.652313    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b","openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc","kube-system/konnectivity-agent-wc9ll","openshift-insights/insights-runtime-extractor-9hsfk","openshift-network-diagnostics/network-check-target-7t4gd","openshift-ovn-kubernetes/ovnkube-node-pcn6c","kube-system/global-pull-secret-syncer-ljkpr","openshift-cluster-node-tuning-operator/tuned-pbvpg","openshift-ingress-canary/ingress-canary-xht6q","openshift-multus/multus-additional-cni-plugins-dcf5t","openshift-monitoring/prometheus-k8s-0","openshift-multus/multus-hkp7r","openshift-multus/network-metrics-daemon-x2887","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal","openshift-image-registry/node-ca-c65l7","openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj","openshift-dns/dns-default-qw24r","openshift-monitoring/node-exporter-gg8vl","openshift-network-operator/iptables-alerter-vkrw9","kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal","openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l","openshift-dns/node-resolver-rwxvq"]
Nov 02 21:35:02.660780 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660759    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.660885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660809    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.660885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660849    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.660885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660869    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.661099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660911    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:35:02.661099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660936    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.661099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660973    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.661099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.660991    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkqw\" (UniqueName: \"kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.661221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661116    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.661221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661157    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:35:02.661221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661177    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83a7a1-7653-4b92-ada5-137be7518e4e-data-volume\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.661221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661184    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.661221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661208    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.661397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661232    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlwx\" (UniqueName: \"kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.661397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661297    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.661397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661355    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.661397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661389    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661414    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661429    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661453    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661471    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661492    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661513    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdsrg\" (UniqueName: \"kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.661546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661535    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661542    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-tmp-dir\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661564    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661591    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661620    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661642    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.661880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.661693    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.662274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662255    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.662327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662297    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.662388 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662369    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.662439 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662402    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.662487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662434    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.662535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662480    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.662535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662511    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.662594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662539    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-etc-tuned\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.662594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662555    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.662594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662582    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.662677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662606    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sw4\" (UniqueName: \"kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.662677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662631    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.662734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662720    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.662763 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662748    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.662801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662789    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.662830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662807    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.662863 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662830    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6bz\" (UniqueName: \"kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.662891 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.662918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662889    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.662918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662905    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662921    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662972    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzxs\" (UniqueName: \"kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:35:02.663010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.662994    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.663143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663016    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4268\" (UniqueName: \"kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.663143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663023    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4e1a1df8-8a6a-4136-9444-9eb795619aed-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.663205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663155    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663173    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663186    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663202    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9gfl\" (UniqueName: \"kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663226    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663266    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663294    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663323    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663351    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597wm\" (UniqueName: \"kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.663418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663377    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663422    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663444    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663485    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663519    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/103a29ef-eca9-4925-93df-023ee970d7f7-tmp\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663563    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663588    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.663643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663639    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663665    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663679    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b3fb92a-6bfe-4d2f-968e-81741b324547-tmp-dir\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663681    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kr9g\" (UniqueName: \"kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663739    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663757    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl9zx\" (UniqueName: \"kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663773    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663789    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.663894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663832    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdfv\" (UniqueName: \"kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663894    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663932    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.663987    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664012    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664028    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664060    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664102    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664123    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.664185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664152    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664192    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664208    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664227    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvs7m\" (UniqueName: \"kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664243    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664278    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664390    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664415    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664443    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.664516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664503    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664536    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664555    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664569    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664582    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664596    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.664783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.664611    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.706125 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706101    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qbt7b\""
Nov 02 21:35:02.706202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706148    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 21:35:02.706202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706167    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 21:35:02.706202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706173    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 21:35:02.706202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706173    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f7c29\""
Nov 02 21:35:02.706341 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706205    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-95jdp\""
Nov 02 21:35:02.706341 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.706214    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2mch2\""
Nov 02 21:35:02.708376 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.708364    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-b8lsz\""
Nov 02 21:35:02.711496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.711481    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab35d4ec-3432-43b7-adca-34097c507ae1-agent-certs\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:35:02.711786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.711769    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dsxz6\""
Nov 02 21:35:02.712063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712040    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jhcsj\""
Nov 02 21:35:02.712063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712020    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 21:35:02.712180 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712089    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.712180 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712100    5580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:35:02.712180 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:02.712168    5580 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.712392 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712378    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-7clxp\""
Nov 02 21:35:02.712471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712455    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.712525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712497    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.712525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712513    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.712624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712611    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovn-node-metrics-cert\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.712676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712633    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 21:35:02.712676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712656    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.712676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712660    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.712676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712675    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.712824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.712637    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.713120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713101    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 21:35:02.713211 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713197    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.713280 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713268    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.713329 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713300    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 21:35:02.713409 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713378    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.713467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713423    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-env-overrides\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.713511 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713488    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 21:35:02.713626 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713608    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.713669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713633    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 21:35:02.713711 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713686    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 21:35:02.713786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713767    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.713915 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713829    5580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 21:35:02.713915 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:02.713871    5580 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-74.us-west-1.compute.internal"
Nov 02 21:35:02.713915 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.713894    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 21:35:02.716719 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.716694    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.717098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.717083    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-cni-binary-copy\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.721408 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.721390    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-config\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.721525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.721513    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-daemon-config\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.721945 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.721934    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab35d4ec-3432-43b7-adca-34097c507ae1-konnectivity-ca\") pod \"konnectivity-agent-wc9ll\" (UID: \"ab35d4ec-3432-43b7-adca-34097c507ae1\") " pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:35:02.723054 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.723039    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e88de0d5-7833-4038-8119-201c5fff01f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.723119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.723105    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-ovnkube-script-lib\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.725800 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.725784    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nr5z4\""
Nov 02 21:35:02.726018 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726002    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.726164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726153    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 21:35:02.726214 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726177    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 21:35:02.726268 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726231    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 21:35:02.726323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726309    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.726369 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726314    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-fbz7r\""
Nov 02 21:35:02.726369 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726342    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.726369 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726346    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5rwnh\""
Nov 02 21:35:02.726369 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726360    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.726515 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726360    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sb4td\""
Nov 02 21:35:02.726515 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726443    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 21:35:02.726515 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.726481    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zd85x\""
Nov 02 21:35:02.731635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.731620    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c18553f-6a3e-4c2a-a4e1-424c05f27216-metrics-certs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:35:02.734196 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.734181    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4905999a-0718-4c3d-8cfd-abe292d5e01c-serviceca\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.735018 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.735005    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b35ac1d5-682e-4eab-9a5b-a0675b542860-iptables-alerter-script\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.735164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.735150    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e177cbc-5f97-45c8-9714-73510ec5597b-original-pull-secret\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.750048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750029    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.750114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750105    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 21:35:02.750146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750113    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.750146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750119    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 21:35:02.750393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750378    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.750487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750469    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-plzkt\""
Nov 02 21:35:02.750559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750485    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 02 21:35:02.750559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750495    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 21:35:02.750627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750586    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 21:35:02.750669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750626    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.750716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750704    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 21:35:02.750761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750738    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 21:35:02.750761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.750744    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 02 21:35:02.751035 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751023    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 02 21:35:02.751096 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751042    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 21:35:02.751163 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751150    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-stcz5\""
Nov 02 21:35:02.751191 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751157    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 21:35:02.751346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751331    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-z597s\""
Nov 02 21:35:02.751433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751422    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 21:35:02.751466 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751432    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xsz5x\""
Nov 02 21:35:02.751709 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751696    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 02 21:35:02.751760 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751722    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 21:35:02.751760 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751734    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fwwgs\""
Nov 02 21:35:02.751760 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751740    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q2lsm\""
Nov 02 21:35:02.751878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751815    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b3fb92a-6bfe-4d2f-968e-81741b324547-config-volume\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.751878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751847    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 02 21:35:02.751939 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751881    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 02 21:35:02.751939 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.751882    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 21:35:02.752108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.752092    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7gg4d807kkhps\""
Nov 02 21:35:02.752145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.752124    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.752176 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.752160    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83a7a1-7653-4b92-ada5-137be7518e4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.752176 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.752162    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.753366 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.753351    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.753572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.753560    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 02 21:35:02.753895 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.753885    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 02 21:35:02.754184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.754170    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b3fb92a-6bfe-4d2f-968e-81741b324547-metrics-tls\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.754220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.754199    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 02 21:35:02.754280 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.754254    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aec2dacc-d493-48f3-9bec-45e232e24e4c-cert\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:35:02.754339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.754325    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.759936 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.759920    5580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 21:35:02.761149 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761136    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e1a1df8-8a6a-4136-9444-9eb795619aed-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.761566 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761552    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 02 21:35:02.761635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761597    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 02 21:35:02.761635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761608    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 02 21:35:02.761635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761608    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 02 21:35:02.761734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761599    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 02 21:35:02.761734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.761643    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 02 21:35:02.762109 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762093    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 02 21:35:02.762160 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762095    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 02 21:35:02.762160 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762094    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6b6c5\""
Nov 02 21:35:02.762297 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762283    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 02 21:35:02.762325 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762283    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 02 21:35:02.762459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762443    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 02 21:35:02.762505 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762464    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 02 21:35:02.762505 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762476    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 02 21:35:02.762505 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762494    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 02 21:35:02.762597 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.762506    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-51ss0e74f2ivr\""
Nov 02 21:35:02.764812 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764799    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:35:02.764850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764819    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.764850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764835    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764851    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764868    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764888    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764914    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764939    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxgq\" (UniqueName: \"kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.764943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764942    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-cnibin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.764978    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765005    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765029    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765055    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765082    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765090    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765097    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765052    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-conf\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765104    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765131    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-textfile\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765142    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765100    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765146    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765153    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-registration-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765144    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.765233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765207    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765246    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765277    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765298    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765329    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765350    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765376    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765376    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765410    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765413    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-hosts-file\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765429    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765444    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765448    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-socket-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765459    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765450    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-dbus\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765481    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765493    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-sys\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765496    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765513    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765520    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b35ac1d5-682e-4eab-9a5b-a0675b542860-host-slash\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765525    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4905999a-0718-4c3d-8cfd-abe292d5e01c-host\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-systemd\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765561    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e177cbc-5f97-45c8-9714-73510ec5597b-kubelet-config\") pod \"global-pull-secret-syncer-ljkpr\" (UID: \"2e177cbc-5f97-45c8-9714-73510ec5597b\") " pod="kube-system/global-pull-secret-syncer-ljkpr"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765572    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjrhh\" (UniqueName: \"kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765603    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765611    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765642    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-conf-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765709    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765730    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765748    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765772    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.765803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765784    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-log-socket\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765797    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765829    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3be53397-eac6-47b3-8b63-faa5479e1f3b-tmp\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765861    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765879    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765849    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-node-log\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765904    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765923    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765937    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-grpc-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765946    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765951    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-cnibin\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.765986    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766009    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-multus-certs\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766018    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766045    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-run\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766050    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766077    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766103    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766127    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766135    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-multus\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766139    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766155    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766173    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-netd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766178    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-device-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766189    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766186    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766225    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766229    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-metrics-client-ca\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766250    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j452n\" (UniqueName: \"kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766266    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766277    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766280    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-ovn-kubernetes\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766308    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-accelerators-collector-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766359    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766390    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766422    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766440    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766437    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766459    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766472    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e88de0d5-7833-4038-8119-201c5fff01f5-system-cni-dir\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766498    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766514    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-var-lib-kubelet\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766514    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766527    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-hostroot\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766553    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766562    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-host\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766583    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766600    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766616    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766631    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766653    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766687    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766693    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-multus-socket-dir-parent\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766711    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766735    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766757    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766767    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766775    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-ovn\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766783    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysctl-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766736    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766803    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766842    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766852    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f84beb-14a1-406c-924e-3896b29f6564-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766878    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3f84beb-14a1-406c-924e-3896b29f6564-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766893    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766907    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766918    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766920    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-netns\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766918    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.766898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766943    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766976    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-kubelet\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766981    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767010    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767015    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-cni-bin\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.766982    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767035    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767055    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-var-lib-kubelet\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767059    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767075    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83a7a1-7653-4b92-ada5-137be7518e4e-crio-socket\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767085    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767118    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-etc-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767121    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-sysconfig\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767125    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767143    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767172    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767173    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767186    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767211    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767220    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-systemd-units\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767225    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767242    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-run-systemd\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767263    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767232    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767277    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767264    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-slash\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767294    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767316    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767331    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767347    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767371    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-lib-modules\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767373    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767403    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-cni-bin\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767388    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-sys-fs\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767415    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767440    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-kubernetes\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767446    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767475    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-tls\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767486    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p96zw\" (UniqueName: \"kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767493    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-system-cni-dir\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767514    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767518    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767541    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767565    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767566    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-var-lib-openvswitch\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767589    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767615    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767619    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be53397-eac6-47b3-8b63-faa5479e1f3b-metrics-client-ca\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767639    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767651    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/103a29ef-eca9-4925-93df-023ee970d7f7-etc-modprobe-d\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767684    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-etc-kubernetes\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767706    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767735    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767755    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767776    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767795    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-host-run-netns\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767804    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-tls\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767837    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/942b2063-cdbd-49a9-bb85-21053e8f15c6-host-run-k8s-cni-cncf-io\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.768972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.767915    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be53397-eac6-47b3-8b63-faa5479e1f3b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.775189 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.775175    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 02 21:35:02.775726 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.775713    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bzpj6\""
Nov 02 21:35:02.775762 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.775743    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776251    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776630    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzxs\" (UniqueName: \"kubernetes.io/projected/3c18553f-6a3e-4c2a-a4e1-424c05f27216-kube-api-access-qkzxs\") pod \"network-metrics-daemon-x2887\" (UID: \"3c18553f-6a3e-4c2a-a4e1-424c05f27216\") " pod="openshift-multus/network-metrics-daemon-x2887"
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776650    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6bz\" (UniqueName: \"kubernetes.io/projected/b35ac1d5-682e-4eab-9a5b-a0675b542860-kube-api-access-ns6bz\") pod \"iptables-alerter-vkrw9\" (UID: \"b35ac1d5-682e-4eab-9a5b-a0675b542860\") " pod="openshift-network-operator/iptables-alerter-vkrw9"
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776651    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlwx\" (UniqueName: \"kubernetes.io/projected/103a29ef-eca9-4925-93df-023ee970d7f7-kube-api-access-hhlwx\") pod \"tuned-pbvpg\" (UID: \"103a29ef-eca9-4925-93df-023ee970d7f7\") " pod="openshift-cluster-node-tuning-operator/tuned-pbvpg"
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776650    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl9zx\" (UniqueName: \"kubernetes.io/projected/4905999a-0718-4c3d-8cfd-abe292d5e01c-kube-api-access-fl9zx\") pod \"node-ca-c65l7\" (UID: \"4905999a-0718-4c3d-8cfd-abe292d5e01c\") " pod="openshift-image-registry/node-ca-c65l7"
Nov 02 21:35:02.776685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776653    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdsrg\" (UniqueName: \"kubernetes.io/projected/78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c-kube-api-access-vdsrg\") pod \"node-resolver-rwxvq\" (UID: \"78a0fc7f-ccb0-4fd0-bcce-5a13f861c83c\") " pod="openshift-dns/node-resolver-rwxvq"
Nov 02 21:35:02.776933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776698    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkqw\" (UniqueName: \"kubernetes.io/projected/942b2063-cdbd-49a9-bb85-21053e8f15c6-kube-api-access-gbkqw\") pod \"multus-hkp7r\" (UID: \"942b2063-cdbd-49a9-bb85-21053e8f15c6\") " pod="openshift-multus/multus-hkp7r"
Nov 02 21:35:02.776933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776651    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597wm\" (UniqueName: \"kubernetes.io/projected/e88de0d5-7833-4038-8119-201c5fff01f5-kube-api-access-597wm\") pod \"multus-additional-cni-plugins-dcf5t\" (UID: \"e88de0d5-7833-4038-8119-201c5fff01f5\") " pod="openshift-multus/multus-additional-cni-plugins-dcf5t"
Nov 02 21:35:02.776933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.776927    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj598\" (UniqueName: \"kubernetes.io/projected/2f254a4a-36cb-4152-8c7e-b9a1c441c6d8-kube-api-access-zj598\") pod \"network-check-target-7t4gd\" (UID: \"2f254a4a-36cb-4152-8c7e-b9a1c441c6d8\") " pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:35:02.777270 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777248    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sw4\" (UniqueName: \"kubernetes.io/projected/7d83a7a1-7653-4b92-ada5-137be7518e4e-kube-api-access-59sw4\") pod \"insights-runtime-extractor-9hsfk\" (UID: \"7d83a7a1-7653-4b92-ada5-137be7518e4e\") " pod="openshift-insights/insights-runtime-extractor-9hsfk"
Nov 02 21:35:02.777349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777284    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kr9g\" (UniqueName: \"kubernetes.io/projected/1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9-kube-api-access-7kr9g\") pod \"aws-ebs-csi-driver-node-mmwlc\" (UID: \"1ef94b9d-be65-4a3d-a03a-2ad5ffd5a7e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mmwlc"
Nov 02 21:35:02.777349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777305    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9gfl\" (UniqueName: \"kubernetes.io/projected/3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7-kube-api-access-c9gfl\") pod \"ovnkube-node-pcn6c\" (UID: \"3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7\") " pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:02.777452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777380    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4268\" (UniqueName: \"kubernetes.io/projected/1b3fb92a-6bfe-4d2f-968e-81741b324547-kube-api-access-c4268\") pod \"dns-default-qw24r\" (UID: \"1b3fb92a-6bfe-4d2f-968e-81741b324547\") " pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:02.777452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777412    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.777623 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777605    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvs7m\" (UniqueName: \"kubernetes.io/projected/aec2dacc-d493-48f3-9bec-45e232e24e4c-kube-api-access-mvs7m\") pod \"ingress-canary-xht6q\" (UID: \"aec2dacc-d493-48f3-9bec-45e232e24e4c\") " pod="openshift-ingress-canary/ingress-canary-xht6q"
Nov 02 21:35:02.777673 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.777611    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdfv\" (UniqueName: \"kubernetes.io/projected/4e1a1df8-8a6a-4136-9444-9eb795619aed-kube-api-access-shdfv\") pod \"kube-state-metrics-5dc8d7d89b-f5w8b\" (UID: \"4e1a1df8-8a6a-4136-9444-9eb795619aed\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-f5w8b"
Nov 02 21:35:02.779266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.779250    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.785731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.785711    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.819375 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.819327    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxgq\" (UniqueName: \"kubernetes.io/projected/459afbc7-52e2-42f9-98f0-b157dfc74671-kube-api-access-nlxgq\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.820249 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.820227    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j452n\" (UniqueName: \"kubernetes.io/projected/e58fab09-a42f-4f63-96a9-cfe58f7ed15c-kube-api-access-j452n\") pod \"monitoring-plugin-77f4f76594-tvxmt\" (UID: \"e58fab09-a42f-4f63-96a9-cfe58f7ed15c\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:35:02.820326 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.820273    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjrhh\" (UniqueName: \"kubernetes.io/projected/3be53397-eac6-47b3-8b63-faa5479e1f3b-kube-api-access-mjrhh\") pod \"thanos-querier-74fc7f6c58-k7tvj\" (UID: \"3be53397-eac6-47b3-8b63-faa5479e1f3b\") " pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:02.820911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.820896    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"prometheus-k8s-0\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:02.820993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.820977    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p96zw\" (UniqueName: \"kubernetes.io/projected/a3f84beb-14a1-406c-924e-3896b29f6564-kube-api-access-p96zw\") pod \"openshift-state-metrics-7bd7489957-sxq8l\" (UID: \"a3f84beb-14a1-406c-924e-3896b29f6564\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sxq8l"
Nov 02 21:35:02.868731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868711    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.868774 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868738    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.868774 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868754    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.868841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868816    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-root\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.868841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868816    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-node-exporter-wtmp\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:02.868894 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:02.868820    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/459afbc7-52e2-42f9-98f0-b157dfc74671-sys\") pod \"node-exporter-gg8vl\" (UID: \"459afbc7-52e2-42f9-98f0-b157dfc74671\") " pod="openshift-monitoring/node-exporter-gg8vl"
Nov 02 21:35:03.603331 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:03.603305    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-tvxmt"
Nov 02 21:35:05.056142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:05.056110    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qw24r"
Nov 02 21:35:05.310337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:05.310266    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj"
Nov 02 21:35:06.932764 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:06.932723    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:06.947656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:06.947625    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:10.881702 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:10.881673    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:11.010019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:11.009993    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:35:12.063127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:12.063051    5580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-74.us-west-1.compute.internal" event="NodeReady"
Nov 02 21:35:12.548294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:12.548264    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:12.548505 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:12.548492    5580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 21:35:15.687729 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:15.687698    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pcn6c"
Nov 02 21:35:23.173431 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:23.173393    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wc9ll"
Nov 02 21:35:26.308161 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:26.308125    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:35:31.012773 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:31.012744    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7t4gd"
Nov 02 21:35:39.782001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.781938    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:39.782618 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782371    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="prometheus" containerID="cri-o://348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5" gracePeriod=600
Nov 02 21:35:39.782679 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782607    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="config-reloader" containerID="cri-o://165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc" gracePeriod=600
Nov 02 21:35:39.782679 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782586    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-web" containerID="cri-o://621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13" gracePeriod=600
Nov 02 21:35:39.782801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782731    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561" gracePeriod=600
Nov 02 21:35:39.782801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782754    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy" containerID="cri-o://8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88" gracePeriod=600
Nov 02 21:35:39.783458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:39.782836    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="thanos-sidecar" containerID="cri-o://042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32" gracePeriod=600
Nov 02 21:35:40.024794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.024770    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:40.149450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149381    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149429    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149466    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149503    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149531    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149576    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149604    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149648    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.149878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149677    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.150041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.149889    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.150907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.150874    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.151318 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151283    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151330    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151361    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151385    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151414    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151451    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151483    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.151559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.151512    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.152362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.152134    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.153860 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.153839    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.154041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.153893    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.154041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.153932    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") pod \"880e1a9f-5711-436e-938f-619f3e692a56\" (UID: \"880e1a9f-5711-436e-938f-619f3e692a56\") "
Nov 02 21:35:40.154243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.154102    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.154243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.154200    5580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.154243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.154220    5580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.154243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.154237    5580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.156158 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.156113    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:40.156390 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.156359    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.156757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.156736    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.156911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.156859    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:40.157455 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.157414    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:35:40.157867 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.157831    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:35:40.157950 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.157892    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.158047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.157978    5580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.158047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.158005    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.158170 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.158103    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.158243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.158218    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.158335 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.158311    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.158386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.158358    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.159249 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.159225    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.159323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.159289    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.160215 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.160195    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "kube-api-access-fmd99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:35:40.165303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.165277    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config" pod "880e1a9f-5711-436e-938f-619f3e692a56" (UID: "880e1a9f-5711-436e-938f-619f3e692a56"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:35:40.258691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258668    5580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-web-config\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258690    5580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258702    5580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-tls-assets\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258712    5580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-config\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258720    5580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-config-out\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258730    5580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-metrics-client-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258739    5580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258754    5580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/880e1a9f-5711-436e-938f-619f3e692a56-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258763    5580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-grpc-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258771    5580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258779    5580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258788    5580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-metrics-client-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258796    5580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258806    5580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/880e1a9f-5711-436e-938f-619f3e692a56-prometheus-k8s-db\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258814    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmd99\" (UniqueName: \"kubernetes.io/projected/880e1a9f-5711-436e-938f-619f3e692a56-kube-api-access-fmd99\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.258824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.258822    5580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/880e1a9f-5711-436e-938f-619f3e692a56-secret-kube-rbac-proxy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:35:40.780768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780736    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561" exitCode=0
Nov 02 21:35:40.780768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780762    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88" exitCode=0
Nov 02 21:35:40.780768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780769    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13" exitCode=0
Nov 02 21:35:40.780768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780776    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32" exitCode=0
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780783    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc" exitCode=0
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780789    5580 generic.go:356] "Generic (PLEG): container finished" podID="880e1a9f-5711-436e-938f-619f3e692a56" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5" exitCode=0
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780820    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"}
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780888    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"}
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780826    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780923    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.780915    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"}
Nov 02 21:35:40.781119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.781101    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"}
Nov 02 21:35:40.781502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.781150    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"}
Nov 02 21:35:40.781502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.781180    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"}
Nov 02 21:35:40.781502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.781213    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"880e1a9f-5711-436e-938f-619f3e692a56","Type":"ContainerDied","Data":"36bf49f9220445c6e3cc61f06ed62275d698006bc96bb8344f9274dd396dffcb"}
Nov 02 21:35:40.788819 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.788716    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.795140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.795128    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.809194 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.809176    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.815160 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.815142    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.821020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.821000    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.827266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.827246    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.833238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.833221    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.833498 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.833480    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.833540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.833505    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.833540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.833535    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.833772 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.833757    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.833829 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.833777    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.833829 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.833790    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.834049 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.834032    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.834146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834059    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.834146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834079    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.834308 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.834291    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.834347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834312    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.834347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834324    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.834517 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.834501    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.834582 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834524    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.834582 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834542    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.834731 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.834713    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.834784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834733    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.834784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.834750    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.835005 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:40.834986    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.835074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835011    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.835074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835030    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.835234 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835215    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.835298 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835235    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.835482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835463    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.835525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835483    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.835685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835669    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.835744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835686    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.835909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835891    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.835971 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.835911    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.836172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836154    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.836224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836173    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.836374 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836356    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.836438 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836374    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.836612 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836594    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.836661 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836613    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.836861 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836844    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.836924 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.836862    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.837092 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837073    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.837143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837094    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.837308 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837290    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.837372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837308    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.837542 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837523    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.837585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837543    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.837752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837736    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.837810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837752    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.837981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837943    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.837981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.837974    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.838182 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838165    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.838246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838183    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.838394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838375    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.838475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838394    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.838572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838558    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.838612 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838572    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.838784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838763    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.838828 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838785    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.839008 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.838991    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.839083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839009    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.839231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839216    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.839277 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839231    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.839433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839416    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.839498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839434    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.839647 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839629    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.839700 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839647    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.839826 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839808    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.839867 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.839825    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.840036 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840017    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.840106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840036    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.840224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840207    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.840262 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840223    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.840437 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840418    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.840477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840437    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.840638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840618    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.840678 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840639    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.840835 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840817    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.840877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.840835    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.841036 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841020    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.841080 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841035    5580 scope.go:122] "RemoveContainer" containerID="3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"
Nov 02 21:35:40.841223 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841206    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561"} err="failed to get container status \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": rpc error: code = NotFound desc = could not find container \"3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561\": container with ID starting with 3b7202625291a876185e5b7f4ddd9d82b1d248522da643473a5818d1767cd561 not found: ID does not exist"
Nov 02 21:35:40.841263 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841223    5580 scope.go:122] "RemoveContainer" containerID="8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"
Nov 02 21:35:40.841423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841405    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88"} err="failed to get container status \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": rpc error: code = NotFound desc = could not find container \"8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88\": container with ID starting with 8aef0d3a5c1c5a2ac4be75437b2b70bfd0ade5b85b7c3f870711e4e8a419be88 not found: ID does not exist"
Nov 02 21:35:40.841475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841423    5580 scope.go:122] "RemoveContainer" containerID="621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"
Nov 02 21:35:40.841618 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841600    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13"} err="failed to get container status \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": rpc error: code = NotFound desc = could not find container \"621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13\": container with ID starting with 621901c74721dee57b1bb71ddf50549c98f688bfe0fbfc73d40b24d50ee96e13 not found: ID does not exist"
Nov 02 21:35:40.841678 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841619    5580 scope.go:122] "RemoveContainer" containerID="042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"
Nov 02 21:35:40.841816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841800    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32"} err="failed to get container status \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": rpc error: code = NotFound desc = could not find container \"042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32\": container with ID starting with 042f89f935a7258cef67fdd8ea174344da8656784fbcc7c440e17c6249a3dd32 not found: ID does not exist"
Nov 02 21:35:40.841868 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.841816    5580 scope.go:122] "RemoveContainer" containerID="165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"
Nov 02 21:35:40.842056 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.842038    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc"} err="failed to get container status \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": rpc error: code = NotFound desc = could not find container \"165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc\": container with ID starting with 165929fefebbd2d02fbb690f55c4ab61b5701110b13ea667c2388cdb0b7ff9dc not found: ID does not exist"
Nov 02 21:35:40.842110 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.842057    5580 scope.go:122] "RemoveContainer" containerID="348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"
Nov 02 21:35:40.842239 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.842221    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5"} err="failed to get container status \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": rpc error: code = NotFound desc = could not find container \"348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5\": container with ID starting with 348535a5120aa26609ed69125c7999fef7f856a52455e4afe8e19be8132742f5 not found: ID does not exist"
Nov 02 21:35:40.842305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.842240    5580 scope.go:122] "RemoveContainer" containerID="be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"
Nov 02 21:35:40.842420 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.842404    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b"} err="failed to get container status \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": rpc error: code = NotFound desc = could not find container \"be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b\": container with ID starting with be2bba155d20f23e7f0a20258feb1fe7ad1998523093f41944729faa2aa60f8b not found: ID does not exist"
Nov 02 21:35:40.944228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.944208    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:40.994328 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994301    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:40.994555 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994541    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:40.994783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994771    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-thanos"
Nov 02 21:35:40.994822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994784    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-thanos"
Nov 02 21:35:40.994822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994799    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="config-reloader"
Nov 02 21:35:40.994822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994804    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="config-reloader"
Nov 02 21:35:40.994822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994814    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="init-config-reloader"
Nov 02 21:35:40.994822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994818    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="init-config-reloader"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994828    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-web"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994832    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-web"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994841    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="prometheus"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994846    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="prometheus"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994853    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="thanos-sidecar"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994857    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="thanos-sidecar"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994863    5580 cpu_manager.go:414] "containerMap: removing container" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994867    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994907    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="config-reloader"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994914    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="init-config-reloader"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994921    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-web"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994926    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="prometheus"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994931    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="thanos-sidecar"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994937    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy"
Nov 02 21:35:40.995047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.994944    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="880e1a9f-5711-436e-938f-619f3e692a56" containerName="kube-rbac-proxy-thanos"
Nov 02 21:35:40.998114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:40.998100    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.050993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.050918    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 02 21:35:41.050993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.050927    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 02 21:35:41.051299 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051248    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 02 21:35:41.051299 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051292    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 02 21:35:41.051458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051297    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 02 21:35:41.051458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051297    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 02 21:35:41.051458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051393    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051900    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051916    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.051910    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bzpj6\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.052179    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.052368    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 02 21:35:41.052666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.052455    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 02 21:35:41.053033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.052728    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 02 21:35:41.053033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.052989    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-51ss0e74f2ivr\""
Nov 02 21:35:41.102552 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.102529    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 02 21:35:41.102552 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.102541    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 02 21:35:41.165677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165657    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165762 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165684    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm2jl\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-kube-api-access-qm2jl\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165762 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165703    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165762 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165726    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165860 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165825    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165860 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165850    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165924 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165866    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.165924 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165887    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166005 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165967    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166005 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.165989    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166075 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166007    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166075 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166026    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166136 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166086    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166166 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166137    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166166 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166153    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166171    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166185    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166201    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166329 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166231    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.166329 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.166271    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.226255 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.226143    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:41.267013 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.266992    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267021    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267037    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267051    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267173    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267211    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267306 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267230    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm2jl\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-kube-api-access-qm2jl\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267306 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267259    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267306 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267290    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267319    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267348    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267370    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267402    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267419    5580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 21:35:41.267668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267425    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267538    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267569    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267607    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267639    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267676    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.267900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267712    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.268021 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.267915    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.268340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.268311    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.268429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.268371    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.268684 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.268664    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.270816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.270089    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.270816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.270120    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.270816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.270651    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.270816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.270695    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.270816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.270731    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.271699    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272156    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272262    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272288    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddb36de-40b5-40e8-a49c-7ddcb7d98550-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272460 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272406    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272460 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272438    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddb36de-40b5-40e8-a49c-7ddcb7d98550-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272797 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272774    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.272889 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.272772    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.273024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.273006    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.273352 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.273333    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddb36de-40b5-40e8-a49c-7ddcb7d98550-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.325120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.325072    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm2jl\" (UniqueName: \"kubernetes.io/projected/bddb36de-40b5-40e8-a49c-7ddcb7d98550-kube-api-access-qm2jl\") pod \"prometheus-k8s-0\" (UID: \"bddb36de-40b5-40e8-a49c-7ddcb7d98550\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:41.590185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.590103    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="880e1a9f-5711-436e-938f-619f3e692a56" path="/var/lib/kubelet/pods/880e1a9f-5711-436e-938f-619f3e692a56/volumes"
Nov 02 21:35:41.613548 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:41.613502    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:42.085178 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:42.085144    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 02 21:35:42.088744 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:35:42.088718    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddb36de_40b5_40e8_a49c_7ddcb7d98550.slice/crio-94f3abefd821cf2891f372404a5979af181829a6d8da1e4708b78be547e4ca2c WatchSource:0}: Error finding container 94f3abefd821cf2891f372404a5979af181829a6d8da1e4708b78be547e4ca2c: Status 404 returned error can't find the container with id 94f3abefd821cf2891f372404a5979af181829a6d8da1e4708b78be547e4ca2c
Nov 02 21:35:42.789794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:42.789760    5580 generic.go:356] "Generic (PLEG): container finished" podID="bddb36de-40b5-40e8-a49c-7ddcb7d98550" containerID="c1935616b426c29957fe4f9817bb7213323877aebf2c098f4cec42b248c429d2" exitCode=0
Nov 02 21:35:42.790004 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:42.789853    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerDied","Data":"c1935616b426c29957fe4f9817bb7213323877aebf2c098f4cec42b248c429d2"}
Nov 02 21:35:42.790004 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:42.789941    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"94f3abefd821cf2891f372404a5979af181829a6d8da1e4708b78be547e4ca2c"}
Nov 02 21:35:43.795922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.795894    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:35:43.796317 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796290    5580 generic.go:356] "Generic (PLEG): container finished" podID="bddb36de-40b5-40e8-a49c-7ddcb7d98550" containerID="0a812116c249d5301caa8709e4cbba3af5b2e87b1e7b21eac3c4ab3ceac1f20b" exitCode=2
Nov 02 21:35:43.796372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796358    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"45f358da1f871edddc18e9aa87be73f61e72e94df0c9f27d2bd0f71b2ec31140"}
Nov 02 21:35:43.796468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796456    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"a9f5d3812b4eef7b0f698fc547417d2ddb07671d9a052f98da92b9790dca6f67"}
Nov 02 21:35:43.796518 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796499    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"f2ea572bb023cd774b27f9566a572ac07b186138d662b559d04a613354d51238"}
Nov 02 21:35:43.796547 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796537    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"01a009f49c4badcb15608b725fce36d7dbc0e223d7f2dc84b3d96d2eeebf685e"}
Nov 02 21:35:43.796585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796572    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"1de2cd4204a180b190c1a7b1db7b8f8902a64d833153df33bedc1952714530cf"}
Nov 02 21:35:43.796622 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796612    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerDied","Data":"0a812116c249d5301caa8709e4cbba3af5b2e87b1e7b21eac3c4ab3ceac1f20b"}
Nov 02 21:35:43.796885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:43.796871    5580 scope.go:122] "RemoveContainer" containerID="0a812116c249d5301caa8709e4cbba3af5b2e87b1e7b21eac3c4ab3ceac1f20b"
Nov 02 21:35:44.802711 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:44.802685    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:35:44.803153 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:44.803131    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddb36de-40b5-40e8-a49c-7ddcb7d98550","Type":"ContainerStarted","Data":"21143ad9364642965c933f15245bb19bbdf730588f8ed331c5a96d93fe1c2a72"}
Nov 02 21:35:45.024423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:45.024367    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.007046836 podStartE2EDuration="5.024351891s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.305055ms" isStatefulPod=false podCreationTimestamp="2026-11-02 21:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:35:44.974677176 +0000 UTC m=+43.491172948" watchObservedRunningTime="2026-11-02 21:35:45.024351891 +0000 UTC m=+43.540847657"
Nov 02 21:35:46.613594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:46.613560    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:47.937259 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:47.937230    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:47.954419 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:47.954390    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:48.835379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:35:48.835353    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 02 21:35:56.307593 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:35:56.307554    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:36:26.308667 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:36:26.308629    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:36:56.308823 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:36:56.308783    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:37:26.308766 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:37:26.308726    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:37:56.308198 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:37:56.308161    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:38:26.307919 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:38:26.307886    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:38:26.839752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.839718    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"]
Nov 02 21:38:26.841901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.841881    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:26.894207 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.894176    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 02 21:38:26.943923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.943888    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-pd7dp\""
Nov 02 21:38:26.944090 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.943992    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 02 21:38:26.944418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.944398    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"]
Nov 02 21:38:26.990508 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.990472    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp5gg\" (UniqueName: \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:26.990707 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.990547    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:26.990707 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:26.990618    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.091450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.091359    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.091450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.091399    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.091450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.091443    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp5gg\" (UniqueName: \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.091806 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.091786    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.091871 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.091814    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.147577 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.147544    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp5gg\" (UniqueName: \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.153346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.153320    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:27.518317 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.518285    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"]
Nov 02 21:38:27.521663 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:38:27.521635    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600760e2_0518_4538_aaec_88cd61829eb9.slice/crio-d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd WatchSource:0}: Error finding container d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd: Status 404 returned error can't find the container with id d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd
Nov 02 21:38:27.523853 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:27.523834    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:38:28.348975 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:28.348920    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g" event={"ID":"600760e2-0518-4538-aaec-88cd61829eb9","Type":"ContainerStarted","Data":"d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd"}
Nov 02 21:38:31.284190 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.284135    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"]
Nov 02 21:38:31.286620 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.286595    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.338651 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.338620    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-skfkh\""
Nov 02 21:38:31.369638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.369604    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 21:38:31.375991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.375945    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 21:38:31.375991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.375945    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 21:38:31.389279 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.389247    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"]
Nov 02 21:38:31.424991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.424937    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwh2w\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-kube-api-access-bwh2w\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425059    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca908a18-62ad-40f8-9929-190590cb62ea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425098    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425131    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425257 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425242    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425331 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425311    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425375 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425344    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca908a18-62ad-40f8-9929-190590cb62ea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425375 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425369    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.425447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.425420    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.525911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.525867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwh2w\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-kube-api-access-bwh2w\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.525911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.525917    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca908a18-62ad-40f8-9929-190590cb62ea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.525996    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526089    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526146    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526180    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526205    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca908a18-62ad-40f8-9929-190590cb62ea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526236    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526300    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526462    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526602    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526627    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.526832 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.526807    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.527080 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.527056    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca908a18-62ad-40f8-9929-190590cb62ea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.528457 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.528431    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca908a18-62ad-40f8-9929-190590cb62ea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.528727 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.528709    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca908a18-62ad-40f8-9929-190590cb62ea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.582918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.582828    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.583067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.582935    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwh2w\" (UniqueName: \"kubernetes.io/projected/ca908a18-62ad-40f8-9929-190590cb62ea-kube-api-access-bwh2w\") pod \"openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd\" (UID: \"ca908a18-62ad-40f8-9929-190590cb62ea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:31.598972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:31.598929    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:32.022680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:32.022650    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"]
Nov 02 21:38:32.025929 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:38:32.025904    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca908a18_62ad_40f8_9929_190590cb62ea.slice/crio-c001912bbec8b131af528f81dac079b435e9b515095ec9e72f8714b8da805026 WatchSource:0}: Error finding container c001912bbec8b131af528f81dac079b435e9b515095ec9e72f8714b8da805026: Status 404 returned error can't find the container with id c001912bbec8b131af528f81dac079b435e9b515095ec9e72f8714b8da805026
Nov 02 21:38:32.368516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:32.368428    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd" event={"ID":"ca908a18-62ad-40f8-9929-190590cb62ea","Type":"ContainerStarted","Data":"c001912bbec8b131af528f81dac079b435e9b515095ec9e72f8714b8da805026"}
Nov 02 21:38:33.373977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:33.373869    5580 generic.go:356] "Generic (PLEG): container finished" podID="600760e2-0518-4538-aaec-88cd61829eb9" containerID="2a166ef7989f075ae75562edfa5806cd188758b6c5fbdc85873bddb8cefad7cd" exitCode=0
Nov 02 21:38:33.373977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:33.373918    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g" event={"ID":"600760e2-0518-4538-aaec-88cd61829eb9","Type":"ContainerDied","Data":"2a166ef7989f075ae75562edfa5806cd188758b6c5fbdc85873bddb8cefad7cd"}
Nov 02 21:38:36.387915 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:36.387882    5580 generic.go:356] "Generic (PLEG): container finished" podID="600760e2-0518-4538-aaec-88cd61829eb9" containerID="f7276984d282b9a3d12d6f3b823f94555be86ab55448d82bc2307039f38609d9" exitCode=0
Nov 02 21:38:36.388394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:36.387925    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g" event={"ID":"600760e2-0518-4538-aaec-88cd61829eb9","Type":"ContainerDied","Data":"f7276984d282b9a3d12d6f3b823f94555be86ab55448d82bc2307039f38609d9"}
Nov 02 21:38:36.507559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:36.507529    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:38:36.507655 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:36.507591    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:38:36.507655 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:36.507641    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:38:37.393810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:37.393768    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd" event={"ID":"ca908a18-62ad-40f8-9929-190590cb62ea","Type":"ContainerStarted","Data":"0a99cd085d6535e9d0f87c9fd60d1d4a7aa03d52b1662d709791760616247990"}
Nov 02 21:38:37.599840 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:37.599804    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:37.604819 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:37.604797    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:37.610528 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:37.610122    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd" podStartSLOduration=2.131443707 podStartE2EDuration="6.610104965s" totalImagesPullingTime="4.478661258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:38:37.56034564 +0000 UTC m=+216.076841413" watchObservedRunningTime="2026-11-02 21:38:37.610104965 +0000 UTC m=+216.126600730"
Nov 02 21:38:38.398033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:38.398004    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:38.399254 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:38.399232    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd"
Nov 02 21:38:43.419433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:43.419396    5580 generic.go:356] "Generic (PLEG): container finished" podID="600760e2-0518-4538-aaec-88cd61829eb9" containerID="a93a867e2a9afce3f99ceaad506c8d36cab5c2eb04cb2e94f9e636af9dc2db44" exitCode=0
Nov 02 21:38:43.419905 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:43.419477    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g" event={"ID":"600760e2-0518-4538-aaec-88cd61829eb9","Type":"ContainerDied","Data":"a93a867e2a9afce3f99ceaad506c8d36cab5c2eb04cb2e94f9e636af9dc2db44"}
Nov 02 21:38:44.538205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.538181    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:44.639087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.639044    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\" (UniqueName: \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\") pod \"600760e2-0518-4538-aaec-88cd61829eb9\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") "
Nov 02 21:38:44.639265 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.639102    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\") pod \"600760e2-0518-4538-aaec-88cd61829eb9\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") "
Nov 02 21:38:44.639265 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.639138    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\") pod \"600760e2-0518-4538-aaec-88cd61829eb9\" (UID: \"600760e2-0518-4538-aaec-88cd61829eb9\") "
Nov 02 21:38:44.639622 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.639598    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle" pod "600760e2-0518-4538-aaec-88cd61829eb9" (UID: "600760e2-0518-4538-aaec-88cd61829eb9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:44.641286 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.641251    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg" pod "600760e2-0518-4538-aaec-88cd61829eb9" (UID: "600760e2-0518-4538-aaec-88cd61829eb9"). InnerVolumeSpecName "kube-api-access-sp5gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:38:44.643748 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.643722    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util" pod "600760e2-0518-4538-aaec-88cd61829eb9" (UID: "600760e2-0518-4538-aaec-88cd61829eb9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:38:44.740332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.740256    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp5gg\" (UniqueName: \"kubernetes.io/projected/600760e2-0518-4538-aaec-88cd61829eb9-kube-api-access-sp5gg\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:38:44.740332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.740287    5580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-util\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:38:44.740332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:44.740297    5580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600760e2-0518-4538-aaec-88cd61829eb9-bundle\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:38:45.428354 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:45.428326    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g"
Nov 02 21:38:45.428524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:45.428325    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nhv7g" event={"ID":"600760e2-0518-4538-aaec-88cd61829eb9","Type":"ContainerDied","Data":"d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd"}
Nov 02 21:38:45.428524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:38:45.428446    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d524349f856d6470bcf9a024ee990815df7f2fcf2f0084309c236a2ca6e60bdd"
Nov 02 21:38:56.309845 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:38:56.309804    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:39:04.965197 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965164    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"]
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965449    5580 cpu_manager.go:414] "containerMap: removing container" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="util"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965461    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="util"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965482    5580 cpu_manager.go:414] "containerMap: removing container" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="extract"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965487    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="extract"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965498    5580 cpu_manager.go:414] "containerMap: removing container" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="pull"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965503    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="pull"
Nov 02 21:39:04.965634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.965550    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="600760e2-0518-4538-aaec-88cd61829eb9" containerName="extract"
Nov 02 21:39:04.967564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:04.967546    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.022012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.021989    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 02 21:39:05.071629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.071606    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 02 21:39:05.071741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.071724    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-fzcbh\""
Nov 02 21:39:05.072513 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.072495    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"]
Nov 02 21:39:05.095081 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.095063    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/119d7496-4692-4e47-983f-ff6046c24ffa-tmp\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.095140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.095092    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766rz\" (UniqueName: \"kubernetes.io/projected/119d7496-4692-4e47-983f-ff6046c24ffa-kube-api-access-766rz\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.195692 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.195666    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/119d7496-4692-4e47-983f-ff6046c24ffa-tmp\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.195797 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.195706    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766rz\" (UniqueName: \"kubernetes.io/projected/119d7496-4692-4e47-983f-ff6046c24ffa-kube-api-access-766rz\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.196123 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.196105    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/119d7496-4692-4e47-983f-ff6046c24ffa-tmp\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.252043 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.251991    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-766rz\" (UniqueName: \"kubernetes.io/projected/119d7496-4692-4e47-983f-ff6046c24ffa-kube-api-access-766rz\") pod \"jobset-operator-674cd59dc5-vp6qm\" (UID: \"119d7496-4692-4e47-983f-ff6046c24ffa\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.292518 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.292498    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"
Nov 02 21:39:05.657851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:05.657784    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm"]
Nov 02 21:39:05.660713 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:05.660689    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod119d7496_4692_4e47_983f_ff6046c24ffa.slice/crio-3e4a235fce8cd82c530535db3421505354bf0c8a2265015feea07041413f476c WatchSource:0}: Error finding container 3e4a235fce8cd82c530535db3421505354bf0c8a2265015feea07041413f476c: Status 404 returned error can't find the container with id 3e4a235fce8cd82c530535db3421505354bf0c8a2265015feea07041413f476c
Nov 02 21:39:06.500877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:06.500848    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm" event={"ID":"119d7496-4692-4e47-983f-ff6046c24ffa","Type":"ContainerStarted","Data":"3e4a235fce8cd82c530535db3421505354bf0c8a2265015feea07041413f476c"}
Nov 02 21:39:08.789152 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.789113    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"]
Nov 02 21:39:08.791340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.791325    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.926726 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.926656    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-tmp\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.926899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.926803    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.927016 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.926891    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.927089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.926950    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.927148 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.927116    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-manifests\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.927207 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.927189    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6sx\" (UniqueName: \"kubernetes.io/projected/c1101578-a6d0-48ac-9366-a86db0d74b02-kube-api-access-jx6sx\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:08.967170 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967136    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 21:39:08.967357 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967337    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 21:39:08.967422 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967374    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 21:39:08.967422 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967394    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 21:39:08.967522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967421    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 21:39:08.967522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:08.967461    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-cf4br\""
Nov 02 21:39:09.023588 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.023561    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"]
Nov 02 21:39:09.027881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.027853    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-tmp\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.027897    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.027939    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.027987    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.028148    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-manifests\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.028189    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6sx\" (UniqueName: \"kubernetes.io/projected/c1101578-a6d0-48ac-9366-a86db0d74b02-kube-api-access-jx6sx\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.028561 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.028529    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-manifests\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.030623 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.030574    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1101578-a6d0-48ac-9366-a86db0d74b02-tmp\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.030799 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.030783    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.031217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.031195    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.031308 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.031227    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c1101578-a6d0-48ac-9366-a86db0d74b02-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.085804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.085734    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6sx\" (UniqueName: \"kubernetes.io/projected/c1101578-a6d0-48ac-9366-a86db0d74b02-kube-api-access-jx6sx\") pod \"opendatahub-operator-controller-manager-67f87dc46d-z5nhf\" (UID: \"c1101578-a6d0-48ac-9366-a86db0d74b02\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.103977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.103927    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:09.595139 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:09.595114    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"]
Nov 02 21:39:09.595979 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:09.595938    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1101578_a6d0_48ac_9366_a86db0d74b02.slice/crio-e3a64dc0dfca2e951aea41e42938c71cbc652148c616b1108a26839a425935cf WatchSource:0}: Error finding container e3a64dc0dfca2e951aea41e42938c71cbc652148c616b1108a26839a425935cf: Status 404 returned error can't find the container with id e3a64dc0dfca2e951aea41e42938c71cbc652148c616b1108a26839a425935cf
Nov 02 21:39:10.516564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:10.516532    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm" event={"ID":"119d7496-4692-4e47-983f-ff6046c24ffa","Type":"ContainerStarted","Data":"3abc9e45a1e1643cec30fa09b14382c62c6b2c2351aea3b005372d0fbf4ca27b"}
Nov 02 21:39:10.517869 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:10.517844    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf" event={"ID":"c1101578-a6d0-48ac-9366-a86db0d74b02","Type":"ContainerStarted","Data":"e3a64dc0dfca2e951aea41e42938c71cbc652148c616b1108a26839a425935cf"}
Nov 02 21:39:10.632288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:10.632091    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vp6qm" podStartSLOduration=2.225411337 podStartE2EDuration="6.632075823s" totalImagesPullingTime="4.406664486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:10.63152779 +0000 UTC m=+249.148023564" watchObservedRunningTime="2026-11-02 21:39:10.632075823 +0000 UTC m=+249.148571592"
Nov 02 21:39:13.530177 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:13.530135    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf" event={"ID":"c1101578-a6d0-48ac-9366-a86db0d74b02","Type":"ContainerStarted","Data":"4fadc686c4e9fa0e4203856b99ac81a2b919ac312ce5b5dcc4cc2309cb21082a"}
Nov 02 21:39:14.535278 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:14.535243    5580 generic.go:356] "Generic (PLEG): container finished" podID="c1101578-a6d0-48ac-9366-a86db0d74b02" containerID="4fadc686c4e9fa0e4203856b99ac81a2b919ac312ce5b5dcc4cc2309cb21082a" exitCode=0
Nov 02 21:39:14.535659 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:14.535330    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf" event={"ID":"c1101578-a6d0-48ac-9366-a86db0d74b02","Type":"ContainerDied","Data":"4fadc686c4e9fa0e4203856b99ac81a2b919ac312ce5b5dcc4cc2309cb21082a"}
Nov 02 21:39:15.541615 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:15.541583    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf" event={"ID":"c1101578-a6d0-48ac-9366-a86db0d74b02","Type":"ContainerStarted","Data":"5ca6e515496d0502aeccffe7c3bcbe49b07f68ccb93cfcb5d66553670f0ebff2"}
Nov 02 21:39:15.541967 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:15.541696    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:15.718426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:15.718336    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf" podStartSLOduration=2.9036225140000003 podStartE2EDuration="7.718322795s" totalImagesPullingTime="4.325062631s" totalInitContainerRuntime="489.63765ms" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:15.718114216 +0000 UTC m=+254.234609990" watchObservedRunningTime="2026-11-02 21:39:15.718322795 +0000 UTC m=+254.234818567"
Nov 02 21:39:26.308061 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:39:26.308020    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:39:26.547672 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:26.547643    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f87dc46d-z5nhf"
Nov 02 21:39:31.393131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.393102    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"]
Nov 02 21:39:31.397171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.397153    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.415604 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.415580    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.415691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.415627    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ck5\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-kube-api-access-v4ck5\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.499054 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.499016    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 02 21:39:31.499054 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.499043    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xh6dq\""
Nov 02 21:39:31.499288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.499250    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 02 21:39:31.500102 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.500087    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"]
Nov 02 21:39:31.516083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.516057    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.516201 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.516108    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ck5\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-kube-api-access-v4ck5\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.572887 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.572864    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.573115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.573095    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ck5\" (UniqueName: \"kubernetes.io/projected/5506cdcd-477f-4bc0-a8e4-d2151fcb51bf-kube-api-access-v4ck5\") pod \"cert-manager-cainjector-5c57bbd4fc-5kd6n\" (UID: \"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:31.711277 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:31.711238    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"
Nov 02 21:39:32.081687 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:32.081606    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n"]
Nov 02 21:39:32.084417 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:32.084390    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5506cdcd_477f_4bc0_a8e4_d2151fcb51bf.slice/crio-c64360e3c6543c78986d67602bf5fdb2c402946fc4171dc5e6f822df0d96e630 WatchSource:0}: Error finding container c64360e3c6543c78986d67602bf5fdb2c402946fc4171dc5e6f822df0d96e630: Status 404 returned error can't find the container with id c64360e3c6543c78986d67602bf5fdb2c402946fc4171dc5e6f822df0d96e630
Nov 02 21:39:32.611116 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:32.611083    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n" event={"ID":"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf","Type":"ContainerStarted","Data":"c64360e3c6543c78986d67602bf5fdb2c402946fc4171dc5e6f822df0d96e630"}
Nov 02 21:39:36.451712 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.451678    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"]
Nov 02 21:39:36.454140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.454118    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.549752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.549722    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.549904 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.549760    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrfg\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-kube-api-access-hsrfg\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.555282 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.555253    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-72vtn\""
Nov 02 21:39:36.556033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.556014    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"]
Nov 02 21:39:36.650869 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.650842    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.650869 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.650877    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrfg\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-kube-api-access-hsrfg\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.708253 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.708184    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.708402 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.708291    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrfg\" (UniqueName: \"kubernetes.io/projected/cb7ac8cf-c178-48ff-922a-8653c7495ef2-kube-api-access-hsrfg\") pod \"cert-manager-webhook-5dbbbf75cc-dc42x\" (UID: \"cb7ac8cf-c178-48ff-922a-8653c7495ef2\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:36.766121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:36.766092    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:37.136469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.136385    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"]
Nov 02 21:39:37.140807 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:37.140779    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb7ac8cf_c178_48ff_922a_8653c7495ef2.slice/crio-35a13829a201079aa5bcde5ebaae02a74db34c915e934496e0f207f3c9a2e4e7 WatchSource:0}: Error finding container 35a13829a201079aa5bcde5ebaae02a74db34c915e934496e0f207f3c9a2e4e7: Status 404 returned error can't find the container with id 35a13829a201079aa5bcde5ebaae02a74db34c915e934496e0f207f3c9a2e4e7
Nov 02 21:39:37.632366 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.632328    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" event={"ID":"cb7ac8cf-c178-48ff-922a-8653c7495ef2","Type":"ContainerStarted","Data":"35a13829a201079aa5bcde5ebaae02a74db34c915e934496e0f207f3c9a2e4e7"}
Nov 02 21:39:37.633793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.633766    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n" event={"ID":"5506cdcd-477f-4bc0-a8e4-d2151fcb51bf","Type":"ContainerStarted","Data":"fb99beae9bb6b310a83d2ef68e60279a7c7f4b29bc6afcbbceb7207a0d46900c"}
Nov 02 21:39:37.748024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.747974    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5kd6n" podStartSLOduration=1.538973851 podStartE2EDuration="6.747942163s" totalImagesPullingTime="5.208968312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:37.747592714 +0000 UTC m=+276.264088482" watchObservedRunningTime="2026-11-02 21:39:37.747942163 +0000 UTC m=+276.264437935"
Nov 02 21:39:37.802134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.802107    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"]
Nov 02 21:39:37.804301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.804286    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.856169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.856144    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 02 21:39:37.856169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.856144    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 02 21:39:37.856355 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.856300    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 02 21:39:37.856396 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.856368    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 02 21:39:37.856488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.856475    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nzqml\""
Nov 02 21:39:37.859792 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.859763    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5l7n\" (UniqueName: \"kubernetes.io/projected/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-kube-api-access-r5l7n\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.859902 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.859800    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.859902 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.859830    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.862294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.862272    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"]
Nov 02 21:39:37.960817 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.960787    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.961023 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.960850    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5l7n\" (UniqueName: \"kubernetes.io/projected/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-kube-api-access-r5l7n\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.961087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.961021    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.961605 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.961582    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:37.963396 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:37.963370    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:38.016915 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.016888    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5l7n\" (UniqueName: \"kubernetes.io/projected/a21ad6c4-0ddc-43c1-affa-d7c953dadc8b-kube-api-access-r5l7n\") pod \"kuadrant-console-plugin-66c4fb575d-pwks5\" (UID: \"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:38.114974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.114921    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"
Nov 02 21:39:38.481379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.481356    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5"]
Nov 02 21:39:38.484657 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:38.484632    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda21ad6c4_0ddc_43c1_affa_d7c953dadc8b.slice/crio-7fe802d007f44f4c19b75321a750284d6b09f2914b3bca14186bc750f8f09691 WatchSource:0}: Error finding container 7fe802d007f44f4c19b75321a750284d6b09f2914b3bca14186bc750f8f09691: Status 404 returned error can't find the container with id 7fe802d007f44f4c19b75321a750284d6b09f2914b3bca14186bc750f8f09691
Nov 02 21:39:38.639618 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.639584    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" event={"ID":"cb7ac8cf-c178-48ff-922a-8653c7495ef2","Type":"ContainerStarted","Data":"da6fd29ebb9a235bd67d3cbbc01723dae11be5f36cb66ab1c4bf8cc045a2915b"}
Nov 02 21:39:38.640065 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.639672    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:38.640919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.640893    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5" event={"ID":"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b","Type":"ContainerStarted","Data":"7fe802d007f44f4c19b75321a750284d6b09f2914b3bca14186bc750f8f09691"}
Nov 02 21:39:38.755951 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:38.755873    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" podStartSLOduration=1.425961531 podStartE2EDuration="2.755859987s" totalImagesPullingTime="1.329898456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:38.754712076 +0000 UTC m=+277.271207830" watchObservedRunningTime="2026-11-02 21:39:38.755859987 +0000 UTC m=+277.272355758"
Nov 02 21:39:39.793294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.793261    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-8hfbd"]
Nov 02 21:39:39.796155 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.796138    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:39.847948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.847925    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-fwgnh\""
Nov 02 21:39:39.877626 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.877602    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jnv\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-kube-api-access-42jnv\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:39.877734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.877648    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-bound-sa-token\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:39.898262 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.898233    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8hfbd"]
Nov 02 21:39:39.978666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.978645    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jnv\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-kube-api-access-42jnv\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:39.978766 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:39.978678    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-bound-sa-token\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:40.034741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.034713    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-bound-sa-token\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:40.034999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.034980    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jnv\" (UniqueName: \"kubernetes.io/projected/e4c7b6e0-7132-41e1-b723-38a22f9172b0-kube-api-access-42jnv\") pod \"cert-manager-597b95cb84-8hfbd\" (UID: \"e4c7b6e0-7132-41e1-b723-38a22f9172b0\") " pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:40.106241 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.106183    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8hfbd"
Nov 02 21:39:40.472044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.472008    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8hfbd"]
Nov 02 21:39:40.478854 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:40.478822    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4c7b6e0_7132_41e1_b723_38a22f9172b0.slice/crio-9dc82cab6f27ad6efb9083dfd6bea75290558ef52171f7ceac75c5deffb2f2e3 WatchSource:0}: Error finding container 9dc82cab6f27ad6efb9083dfd6bea75290558ef52171f7ceac75c5deffb2f2e3: Status 404 returned error can't find the container with id 9dc82cab6f27ad6efb9083dfd6bea75290558ef52171f7ceac75c5deffb2f2e3
Nov 02 21:39:40.658175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.657480    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8hfbd" event={"ID":"e4c7b6e0-7132-41e1-b723-38a22f9172b0","Type":"ContainerStarted","Data":"ffee292c54c1c2555ff310a1f78e713952f018f14483b90e550c88cfcd50cbee"}
Nov 02 21:39:40.658175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.657528    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8hfbd" event={"ID":"e4c7b6e0-7132-41e1-b723-38a22f9172b0","Type":"ContainerStarted","Data":"9dc82cab6f27ad6efb9083dfd6bea75290558ef52171f7ceac75c5deffb2f2e3"}
Nov 02 21:39:40.792837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:40.792727    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-8hfbd" podStartSLOduration=1.792712013 podStartE2EDuration="1.792712013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:40.792615418 +0000 UTC m=+279.309111193" watchObservedRunningTime="2026-11-02 21:39:40.792712013 +0000 UTC m=+279.309207786"
Nov 02 21:39:42.235401 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.235366    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"]
Nov 02 21:39:42.247008 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.246980    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.296552 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.296467    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"]
Nov 02 21:39:42.303682 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.303655    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tmp\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.303841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.303718    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tls-certs\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.303841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.303797    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktdd5\" (UniqueName: \"kubernetes.io/projected/805968c9-e4bc-4a55-8c25-08ddfd3c2234-kube-api-access-ktdd5\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.344922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.344894    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 02 21:39:42.344922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.344894    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 02 21:39:42.345147 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.344996    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xgxvs\""
Nov 02 21:39:42.404913 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.404878    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tls-certs\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.405117 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.404928    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktdd5\" (UniqueName: \"kubernetes.io/projected/805968c9-e4bc-4a55-8c25-08ddfd3c2234-kube-api-access-ktdd5\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.405117 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.405110    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tmp\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.408496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.408473    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tls-certs\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.408649 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.408632    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/805968c9-e4bc-4a55-8c25-08ddfd3c2234-tmp\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.462272 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.462242    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktdd5\" (UniqueName: \"kubernetes.io/projected/805968c9-e4bc-4a55-8c25-08ddfd3c2234-kube-api-access-ktdd5\") pod \"kube-auth-proxy-6d64f9d85b-qz28l\" (UID: \"805968c9-e4bc-4a55-8c25-08ddfd3c2234\") " pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.561383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.561307    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"
Nov 02 21:39:42.952058 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:42.952030    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l"]
Nov 02 21:39:42.955768 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:42.955739    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805968c9_e4bc_4a55_8c25_08ddfd3c2234.slice/crio-3cecec6e7844e5d9497ca6e43948fd02e3498ff604f870eab3dee701b3e49a39 WatchSource:0}: Error finding container 3cecec6e7844e5d9497ca6e43948fd02e3498ff604f870eab3dee701b3e49a39: Status 404 returned error can't find the container with id 3cecec6e7844e5d9497ca6e43948fd02e3498ff604f870eab3dee701b3e49a39
Nov 02 21:39:43.681049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:43.681010    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l" event={"ID":"805968c9-e4bc-4a55-8c25-08ddfd3c2234","Type":"ContainerStarted","Data":"3cecec6e7844e5d9497ca6e43948fd02e3498ff604f870eab3dee701b3e49a39"}
Nov 02 21:39:44.647784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.647752    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x"
Nov 02 21:39:44.813320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.813292    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:39:44.823677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.823655    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:44.852550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.852527    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:39:44.881379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.881360    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 02 21:39:44.929998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.929965    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54rng\" (UniqueName: \"kubernetes.io/projected/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-kube-api-access-54rng\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:44.930168 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:44.930068    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-config-file\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.031460 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.031419    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54rng\" (UniqueName: \"kubernetes.io/projected/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-kube-api-access-54rng\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.031651 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.031473    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-config-file\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.032238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.032206    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-config-file\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.088899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.088872    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54rng\" (UniqueName: \"kubernetes.io/projected/e8897fab-c0b8-4b3c-85e2-4b05af9959cc-kube-api-access-54rng\") pod \"limitador-limitador-7c9b87d574-hkqn9\" (UID: \"e8897fab-c0b8-4b3c-85e2-4b05af9959cc\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.095711 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.095686    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:39:45.145208 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.145180    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:45.996660 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:45.996636    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:39:46.009716 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:39:46.009683    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8897fab_c0b8_4b3c_85e2_4b05af9959cc.slice/crio-6452500d845c4039a929b4228713569f812f07abbc465a552c081040bf96f652 WatchSource:0}: Error finding container 6452500d845c4039a929b4228713569f812f07abbc465a552c081040bf96f652: Status 404 returned error can't find the container with id 6452500d845c4039a929b4228713569f812f07abbc465a552c081040bf96f652
Nov 02 21:39:46.714028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:46.713993    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5" event={"ID":"a21ad6c4-0ddc-43c1-affa-d7c953dadc8b","Type":"ContainerStarted","Data":"b1adc874abe236aa71157b15eadf3e5ec488ebc96b51a633d8302cf2adbff6da"}
Nov 02 21:39:46.715510 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:46.715480    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9" event={"ID":"e8897fab-c0b8-4b3c-85e2-4b05af9959cc","Type":"ContainerStarted","Data":"6452500d845c4039a929b4228713569f812f07abbc465a552c081040bf96f652"}
Nov 02 21:39:46.725085 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:46.725050    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l" event={"ID":"805968c9-e4bc-4a55-8c25-08ddfd3c2234","Type":"ContainerStarted","Data":"81d584c42eeae2903379df29f8e81c5e9207703a4b882ece00df886e85ebd8cf"}
Nov 02 21:39:46.827765 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:46.827712    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwks5" podStartSLOduration=2.678143345 podStartE2EDuration="9.827697604s" totalImagesPullingTime="7.149554259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:46.82694248 +0000 UTC m=+285.343438255" watchObservedRunningTime="2026-11-02 21:39:46.827697604 +0000 UTC m=+285.344193367"
Nov 02 21:39:46.947119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:46.947064    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d64f9d85b-qz28l" podStartSLOduration=1.8866886109999998 podStartE2EDuration="4.947044435s" totalImagesPullingTime="3.060355824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:46.946825873 +0000 UTC m=+285.463321648" watchObservedRunningTime="2026-11-02 21:39:46.947044435 +0000 UTC m=+285.463540209"
Nov 02 21:39:49.739538 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:49.739507    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9" event={"ID":"e8897fab-c0b8-4b3c-85e2-4b05af9959cc","Type":"ContainerStarted","Data":"dfafa478373a4639edf2d1773e4c2fdfdefb17b0f6e13d71403d77716a1db202"}
Nov 02 21:39:49.739935 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:49.739578    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:39:49.858681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:39:49.858621    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9" podStartSLOduration=2.6322621010000002 podStartE2EDuration="5.858600739s" totalImagesPullingTime="3.226338638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:39:49.85806327 +0000 UTC m=+288.374559043" watchObservedRunningTime="2026-11-02 21:39:49.858600739 +0000 UTC m=+288.375096513"
Nov 02 21:39:56.308243 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:39:56.308205    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:40:00.745181 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:00.745149    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"
Nov 02 21:40:01.621440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:01.621414    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:40:01.621624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:01.621456    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:40:01.633540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:01.633513    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:40:01.633670 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:01.633626    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:40:08.314803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.314770    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"]
Nov 02 21:40:08.320121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.320097    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.374319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.374247    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 21:40:08.423910 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.423888    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 21:40:08.431904 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.431880    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w455s\" (UniqueName: \"kubernetes.io/projected/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-kube-api-access-w455s\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.432022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.431913    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.432062 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.432049    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.432100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.432069    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.474820 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.474796    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"]
Nov 02 21:40:08.523897 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.523874    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-fb78q\""
Nov 02 21:40:08.525284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.525257    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 02 21:40:08.532841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.532816    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w455s\" (UniqueName: \"kubernetes.io/projected/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-kube-api-access-w455s\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.532948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.532846    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.532948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.532896    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.532948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.532913    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.533320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.533301    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.533376 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.533359    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.533741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.533724    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.589024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.588977    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w455s\" (UniqueName: \"kubernetes.io/projected/dffd3bb3-8e6a-49da-873a-d182ce1c70c9-kube-api-access-w455s\") pod \"kserve-module-controller-manager-7566cf5fc8-f9srl\" (UID: \"dffd3bb3-8e6a-49da-873a-d182ce1c70c9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:08.631579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:08.631556    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:09.048707 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:09.048476    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"]
Nov 02 21:40:09.050914 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:40:09.050877    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddffd3bb3_8e6a_49da_873a_d182ce1c70c9.slice/crio-a07863f6cd55ad1ed5827fe31748251345ef2a026deba04b0397e847cb837ae3 WatchSource:0}: Error finding container a07863f6cd55ad1ed5827fe31748251345ef2a026deba04b0397e847cb837ae3: Status 404 returned error can't find the container with id a07863f6cd55ad1ed5827fe31748251345ef2a026deba04b0397e847cb837ae3
Nov 02 21:40:09.824807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:09.824767    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl" event={"ID":"dffd3bb3-8e6a-49da-873a-d182ce1c70c9","Type":"ContainerStarted","Data":"a07863f6cd55ad1ed5827fe31748251345ef2a026deba04b0397e847cb837ae3"}
Nov 02 21:40:11.835263 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:11.835227    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl" event={"ID":"dffd3bb3-8e6a-49da-873a-d182ce1c70c9","Type":"ContainerStarted","Data":"5fc4fe59cfaceca2940cf2f8ab5b0d207f81e735c452c27f4ef74457c73966e1"}
Nov 02 21:40:11.835669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:11.835369    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:11.999315 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:11.999261    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl" podStartSLOduration=2.068026858 podStartE2EDuration="3.999247284s" totalImagesPullingTime="1.931220426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:40:11.998639301 +0000 UTC m=+310.515135085" watchObservedRunningTime="2026-11-02 21:40:11.999247284 +0000 UTC m=+310.515743059"
Nov 02 21:40:26.307797 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:40:26.307753    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:40:32.840550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:32.840514    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-f9srl"
Nov 02 21:40:55.518972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.518935    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:40:55.524269 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.524248    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.625720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.625687    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 21:40:55.625911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.625696    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 21:40:55.625911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.625726    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-b76mn\""
Nov 02 21:40:55.625911 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.625793    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 21:40:55.626854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.626831    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:40:55.641222 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.641195    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2cmn\" (UniqueName: \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.641300 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.641228    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.641336 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.641310    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.741895 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.741858    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.742083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.741935    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cmn\" (UniqueName: \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.742083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.741983    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.744329 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.744297    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.744456 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.744349    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.807284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.807166    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2cmn\" (UniqueName: \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\") pod \"maas-controller-579cf7c6d-8khhf\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") " pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.837307 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.837274    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:40:55.941089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.939819    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 21:40:55.943752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:55.943730    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.007090 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.007058    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 21:40:56.044534 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.044494    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.044534 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.044534    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.044730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.044620    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qzkz\" (UniqueName: \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.145921 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.145834    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.145921 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.145875    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.146118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.146004    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qzkz\" (UniqueName: \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.148266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.148248    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.148327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.148278    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.202199 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.202162    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qzkz\" (UniqueName: \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\") pod \"maas-controller-5cc9f7769b-jd2nn\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") " pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.215851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.215825    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:40:56.219017 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:40:56.218989    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ee3aee_280c_4f08_9d90_a43b047fd18b.slice/crio-e3b98725f3bb2a73defc90bed98b5ef04979be55b9a8e8e6df2d697ef2548875 WatchSource:0}: Error finding container e3b98725f3bb2a73defc90bed98b5ef04979be55b9a8e8e6df2d697ef2548875: Status 404 returned error can't find the container with id e3b98725f3bb2a73defc90bed98b5ef04979be55b9a8e8e6df2d697ef2548875
Nov 02 21:40:56.256179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.256153    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:40:56.313773 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:40:56.313712    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:40:56.629133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:56.629089    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 21:40:56.632269 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:40:56.632245    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4bd6e94_7944_4b09_b6ff_340fbae13b9b.slice/crio-cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8 WatchSource:0}: Error finding container cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8: Status 404 returned error can't find the container with id cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8
Nov 02 21:40:57.027992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:57.027939    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8khhf" event={"ID":"f6ee3aee-280c-4f08-9d90-a43b047fd18b","Type":"ContainerStarted","Data":"e3b98725f3bb2a73defc90bed98b5ef04979be55b9a8e8e6df2d697ef2548875"}
Nov 02 21:40:57.029146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:40:57.029121    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerStarted","Data":"cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8"}
Nov 02 21:41:00.046990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:00.046946    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8khhf" event={"ID":"f6ee3aee-280c-4f08-9d90-a43b047fd18b","Type":"ContainerStarted","Data":"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"}
Nov 02 21:41:00.047313 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:00.047039    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:41:00.170264 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:00.170208    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-8khhf" podStartSLOduration=1.742672848 podStartE2EDuration="5.170193708s" totalImagesPullingTime="3.42752086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:00.169443415 +0000 UTC m=+358.685939190" watchObservedRunningTime="2026-11-02 21:41:00.170193708 +0000 UTC m=+358.686689480"
Nov 02 21:41:01.052411 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:01.052371    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerStarted","Data":"e9a0ac87e780a28d3c1de640130351ca97d183e35cbb62a6f8a7d86d378cdb78"}
Nov 02 21:41:01.052889 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:01.052530    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:41:01.189181 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:01.189126    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" podStartSLOduration=2.825942799 podStartE2EDuration="6.189112236s" totalImagesPullingTime="3.363169437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:01.187846799 +0000 UTC m=+359.704342573" watchObservedRunningTime="2026-11-02 21:41:01.189112236 +0000 UTC m=+359.705608029"
Nov 02 21:41:11.054557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:11.054522    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:41:12.059711 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.059632    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:41:12.223629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.223597    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:41:12.224072 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.224035    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-8khhf" podUID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" containerName="manager" containerID="cri-o://f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2" gracePeriod=30
Nov 02 21:41:12.463840 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.463786    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:41:12.596909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.596824    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\") pod \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") "
Nov 02 21:41:12.597085 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.596977    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\") pod \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") "
Nov 02 21:41:12.597085 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.597035    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\" (UniqueName: \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\") pod \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\" (UID: \"f6ee3aee-280c-4f08-9d90-a43b047fd18b\") "
Nov 02 21:41:12.599435 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.599406    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs" pod "f6ee3aee-280c-4f08-9d90-a43b047fd18b" (UID: "f6ee3aee-280c-4f08-9d90-a43b047fd18b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:41:12.599809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.599789    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert" pod "f6ee3aee-280c-4f08-9d90-a43b047fd18b" (UID: "f6ee3aee-280c-4f08-9d90-a43b047fd18b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:41:12.599899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.599815    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn" pod "f6ee3aee-280c-4f08-9d90-a43b047fd18b" (UID: "f6ee3aee-280c-4f08-9d90-a43b047fd18b"). InnerVolumeSpecName "kube-api-access-n2cmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:41:12.698103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.698071    5580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-webhook-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:41:12.698103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.698100    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2cmn\" (UniqueName: \"kubernetes.io/projected/f6ee3aee-280c-4f08-9d90-a43b047fd18b-kube-api-access-n2cmn\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:41:12.698103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.698111    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6ee3aee-280c-4f08-9d90-a43b047fd18b-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:41:12.956683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.956654    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cvqrl"]
Nov 02 21:41:12.957053 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.957041    5580 cpu_manager.go:414] "containerMap: removing container" podUID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" containerName="manager"
Nov 02 21:41:12.957110 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.957054    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" containerName="manager"
Nov 02 21:41:12.957146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.957111    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" containerName="manager"
Nov 02 21:41:12.960025 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:12.960006    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.015019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.014846    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nwvnn\""
Nov 02 21:41:13.015019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.014980    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 02 21:41:13.015925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.015900    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cvqrl"]
Nov 02 21:41:13.101465 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.101430    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9xrm\" (UniqueName: \"kubernetes.io/projected/494cbe8e-607d-4312-9a4f-242ad7930133-kube-api-access-x9xrm\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.101893 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.101533    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494cbe8e-607d-4312-9a4f-242ad7930133-cert\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.103557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.103536    5580 generic.go:356] "Generic (PLEG): container finished" podID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" containerID="f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2" exitCode=0
Nov 02 21:41:13.103625 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.103603    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8khhf"
Nov 02 21:41:13.103664 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.103616    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8khhf" event={"ID":"f6ee3aee-280c-4f08-9d90-a43b047fd18b","Type":"ContainerDied","Data":"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"}
Nov 02 21:41:13.103703 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.103676    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8khhf" event={"ID":"f6ee3aee-280c-4f08-9d90-a43b047fd18b","Type":"ContainerDied","Data":"e3b98725f3bb2a73defc90bed98b5ef04979be55b9a8e8e6df2d697ef2548875"}
Nov 02 21:41:13.103736 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.103708    5580 scope.go:122] "RemoveContainer" containerID="f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"
Nov 02 21:41:13.112743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.112723    5580 scope.go:122] "RemoveContainer" containerID="f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"
Nov 02 21:41:13.113016 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:41:13.112998    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2\": container with ID starting with f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2 not found: ID does not exist" containerID="f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"
Nov 02 21:41:13.113075 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.113025    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2"} err="failed to get container status \"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2\": rpc error: code = NotFound desc = could not find container \"f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2\": container with ID starting with f0ad763692131f09e16146cb4718e3dca69f015f2471ee9ec5889fcaaea0fdb2 not found: ID does not exist"
Nov 02 21:41:13.202099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.202071    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9xrm\" (UniqueName: \"kubernetes.io/projected/494cbe8e-607d-4312-9a4f-242ad7930133-kube-api-access-x9xrm\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.202247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.202139    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494cbe8e-607d-4312-9a4f-242ad7930133-cert\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.204443 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.204421    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/494cbe8e-607d-4312-9a4f-242ad7930133-cert\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.258719 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.258647    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9xrm\" (UniqueName: \"kubernetes.io/projected/494cbe8e-607d-4312-9a4f-242ad7930133-kube-api-access-x9xrm\") pod \"kserve-controller-manager-776cc66d47-cvqrl\" (UID: \"494cbe8e-607d-4312-9a4f-242ad7930133\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.265296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.265270    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:41:13.267378 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.267351    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8khhf"]
Nov 02 21:41:13.272479 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.272460    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:13.291227 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.291200    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-v62z6"]
Nov 02 21:41:13.295269 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.295247    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.351999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.350028    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 02 21:41:13.351999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.350196    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 02 21:41:13.351999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.350861    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pbklg\""
Nov 02 21:41:13.351999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.351262    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-v62z6"]
Nov 02 21:41:13.404338 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.404303    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjbr\" (UniqueName: \"kubernetes.io/projected/86069694-d120-4b17-9a6d-3cb3ac51595b-kube-api-access-jbjbr\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.404484 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.404342    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-cert\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.404484 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.404436    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-metrics-certs\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.505535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.505494    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjbr\" (UniqueName: \"kubernetes.io/projected/86069694-d120-4b17-9a6d-3cb3ac51595b-kube-api-access-jbjbr\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.505535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.505538    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-cert\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.505781 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.505593    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-metrics-certs\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.508081 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.508056    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-metrics-certs\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.508081 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.508081    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86069694-d120-4b17-9a6d-3cb3ac51595b-cert\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.561412 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.561276    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjbr\" (UniqueName: \"kubernetes.io/projected/86069694-d120-4b17-9a6d-3cb3ac51595b-kube-api-access-jbjbr\") pod \"odh-model-controller-769875db56-v62z6\" (UID: \"86069694-d120-4b17-9a6d-3cb3ac51595b\") " pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.591031 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.591002    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6ee3aee-280c-4f08-9d90-a43b047fd18b" path="/var/lib/kubelet/pods/f6ee3aee-280c-4f08-9d90-a43b047fd18b/volumes"
Nov 02 21:41:13.610836 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.610812    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:13.640639 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.640616    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cvqrl"]
Nov 02 21:41:13.643432 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:41:13.643404    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494cbe8e_607d_4312_9a4f_242ad7930133.slice/crio-acc46caa28d42da062ae9425a301447f1fbd81b09cf09c3d0d34a1536e37003c WatchSource:0}: Error finding container acc46caa28d42da062ae9425a301447f1fbd81b09cf09c3d0d34a1536e37003c: Status 404 returned error can't find the container with id acc46caa28d42da062ae9425a301447f1fbd81b09cf09c3d0d34a1536e37003c
Nov 02 21:41:13.974228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:13.974156    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-v62z6"]
Nov 02 21:41:13.976661 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:41:13.976631    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86069694_d120_4b17_9a6d_3cb3ac51595b.slice/crio-ee96d71fbd70c8e0840f03ccc0a71388c5289a335c16fd4e40a8680fd29c2330 WatchSource:0}: Error finding container ee96d71fbd70c8e0840f03ccc0a71388c5289a335c16fd4e40a8680fd29c2330: Status 404 returned error can't find the container with id ee96d71fbd70c8e0840f03ccc0a71388c5289a335c16fd4e40a8680fd29c2330
Nov 02 21:41:14.110181 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:14.110138    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl" event={"ID":"494cbe8e-607d-4312-9a4f-242ad7930133","Type":"ContainerStarted","Data":"acc46caa28d42da062ae9425a301447f1fbd81b09cf09c3d0d34a1536e37003c"}
Nov 02 21:41:14.111453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:14.111423    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-v62z6" event={"ID":"86069694-d120-4b17-9a6d-3cb3ac51595b","Type":"ContainerStarted","Data":"ee96d71fbd70c8e0840f03ccc0a71388c5289a335c16fd4e40a8680fd29c2330"}
Nov 02 21:41:17.129041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:17.128999    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl" event={"ID":"494cbe8e-607d-4312-9a4f-242ad7930133","Type":"ContainerStarted","Data":"77ce6bb1fc259447edb19dc8f7d3c38fb10b004385e2bef6d1cfac48f02ca12f"}
Nov 02 21:41:17.129482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:17.129167    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:17.243920 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:17.243861    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl" podStartSLOduration=1.9747024629999999 podStartE2EDuration="5.243842042s" totalImagesPullingTime="3.269139579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:17.24266759 +0000 UTC m=+375.759163376" watchObservedRunningTime="2026-11-02 21:41:17.243842042 +0000 UTC m=+375.760337815"
Nov 02 21:41:18.136285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:18.136248    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-v62z6" event={"ID":"86069694-d120-4b17-9a6d-3cb3ac51595b","Type":"ContainerStarted","Data":"3fa90b50a36cd189b9b99c4a4f23f7299a83cb222e8ef0e5f049a52f7ab951f5"}
Nov 02 21:41:18.136755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:18.136410    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:18.249204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:18.249146    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-v62z6" podStartSLOduration=1.938521636 podStartE2EDuration="5.249129926s" totalImagesPullingTime="3.31060829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:18.247937858 +0000 UTC m=+376.764433630" watchObservedRunningTime="2026-11-02 21:41:18.249129926 +0000 UTC m=+376.765625693"
Nov 02 21:41:26.308660 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:41:26.308622    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:41:29.142571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:29.142539    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-v62z6"
Nov 02 21:41:41.917607 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:41.917572    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:41:47.915868 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:47.915834    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 21:41:47.922521 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:47.922499    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.016776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.016742    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.016944 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.016800    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.016944 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.016838    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.017044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.016966    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87c4m\" (UniqueName: \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.023819 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.023795    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 21:41:48.023978 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.023841    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 21:41:48.023978 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.023841    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cbb6p\""
Nov 02 21:41:48.023978 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.023845    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 21:41:48.023978 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.023880    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 21:41:48.025182 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.025164    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 21:41:48.117467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.117430    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87c4m\" (UniqueName: \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.117648 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.117488    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.117648 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.117530    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.117648 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.117557    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.118251 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.118217    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.119974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.119939    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.120082 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.120002    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.141120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.141099    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-cvqrl"
Nov 02 21:41:48.173130 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.173103    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87c4m\" (UniqueName: \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\") pod \"maas-api-57cf7b58d8-ppl5p\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") " pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.239106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.239074    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:48.646759 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:48.646667    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 21:41:48.649923 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:41:48.649897    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f737e36_de9c_4020_94a8_4f7070c39163.slice/crio-c586310a44f28138ea041a14cac6a23e0c3af8993a92f3c786c1e778fe8bb0e7 WatchSource:0}: Error finding container c586310a44f28138ea041a14cac6a23e0c3af8993a92f3c786c1e778fe8bb0e7: Status 404 returned error can't find the container with id c586310a44f28138ea041a14cac6a23e0c3af8993a92f3c786c1e778fe8bb0e7
Nov 02 21:41:49.275274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:49.275239    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" event={"ID":"0f737e36-de9c-4020-94a8-4f7070c39163","Type":"ContainerStarted","Data":"c586310a44f28138ea041a14cac6a23e0c3af8993a92f3c786c1e778fe8bb0e7"}
Nov 02 21:41:51.288293 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:51.288253    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" event={"ID":"0f737e36-de9c-4020-94a8-4f7070c39163","Type":"ContainerStarted","Data":"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"}
Nov 02 21:41:51.288784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:51.288395    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:41:51.401739 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:51.401675    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" podStartSLOduration=2.6701460949999998 podStartE2EDuration="4.401655846s" totalImagesPullingTime="1.731509751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:41:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:41:51.401225387 +0000 UTC m=+409.917721160" watchObservedRunningTime="2026-11-02 21:41:51.401655846 +0000 UTC m=+409.918151620"
Nov 02 21:41:56.308521 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:41:56.308483    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:41:57.298337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:41:57.298305    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 21:42:26.308081 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:42:26.308040    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:42:56.308213 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:42:56.308168    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:43:26.309023 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:43:26.308984    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:43:27.620043 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:27.620010    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:43:29.609558 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.609527    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"]
Nov 02 21:43:29.613094 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.613074    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.669060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.669036    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"]
Nov 02 21:43:29.670287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.670263    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 21:43:29.679720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.679693    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrczq\" (UniqueName: \"kubernetes.io/projected/9b51a5e6-f2f4-4349-bf3e-47774cc29115-kube-api-access-wrczq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.679837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.679748    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.679902 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.679851    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.679989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.679917    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.680061 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.680020    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.680105 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.680056    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.717019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.716991    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 02 21:43:29.717019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.717006    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jr9p4\""
Nov 02 21:43:29.717144 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.716991    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 21:43:29.781004 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.780977    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781013    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781055    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781087    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781118    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781191    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrczq\" (UniqueName: \"kubernetes.io/projected/9b51a5e6-f2f4-4349-bf3e-47774cc29115-kube-api-access-wrczq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781438 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781416    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781438    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.781522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.781495    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.783345 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.783320    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b51a5e6-f2f4-4349-bf3e-47774cc29115-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.783497 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.783478    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b51a5e6-f2f4-4349-bf3e-47774cc29115-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.837728 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.837693    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrczq\" (UniqueName: \"kubernetes.io/projected/9b51a5e6-f2f4-4349-bf3e-47774cc29115-kube-api-access-wrczq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc\" (UID: \"9b51a5e6-f2f4-4349-bf3e-47774cc29115\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:29.924932 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:29.924902    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:30.291135 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:30.291035    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"]
Nov 02 21:43:30.293914 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:43:30.293886    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b51a5e6_f2f4_4349_bf3e_47774cc29115.slice/crio-e1cd8532a31c23beafc5c555a749e8a438b7ea46619163e4f7fe84b43762c91b WatchSource:0}: Error finding container e1cd8532a31c23beafc5c555a749e8a438b7ea46619163e4f7fe84b43762c91b: Status 404 returned error can't find the container with id e1cd8532a31c23beafc5c555a749e8a438b7ea46619163e4f7fe84b43762c91b
Nov 02 21:43:30.295312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:30.295295    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:43:30.727277 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:30.727243    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" event={"ID":"9b51a5e6-f2f4-4349-bf3e-47774cc29115","Type":"ContainerStarted","Data":"e1cd8532a31c23beafc5c555a749e8a438b7ea46619163e4f7fe84b43762c91b"}
Nov 02 21:43:32.739544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:32.739502    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" event={"ID":"9b51a5e6-f2f4-4349-bf3e-47774cc29115","Type":"ContainerStarted","Data":"30622e6f97d59987c9d7b642db0c67dd2d506943420aac3775afa27fe8763885"}
Nov 02 21:43:32.852171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:32.852116    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podStartSLOduration=2.018455637 podStartE2EDuration="3.852101385s" totalImagesPullingTime="1.833645748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:32.851991025 +0000 UTC m=+511.368486800" watchObservedRunningTime="2026-11-02 21:43:32.852101385 +0000 UTC m=+511.368597137"
Nov 02 21:43:32.925491 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:32.925451    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:37.268320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.268286    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:43:37.518404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.518328    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"]
Nov 02 21:43:37.522245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.522220    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.624742 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.624703    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 02 21:43:37.625850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.625826    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"]
Nov 02 21:43:37.656219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656179    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.656219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656220    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.656483 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656329    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.656483 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656372    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e267af0e-0894-4514-815d-3bed98989354-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.656549 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656492    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvvs\" (UniqueName: \"kubernetes.io/projected/e267af0e-0894-4514-815d-3bed98989354-kube-api-access-6tvvs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.656549 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.656533    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757612 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757565    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757630    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e267af0e-0894-4514-815d-3bed98989354-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757709    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvvs\" (UniqueName: \"kubernetes.io/projected/e267af0e-0894-4514-815d-3bed98989354-kube-api-access-6tvvs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757745    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757947 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757782    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.757947 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.757841    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.758085 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.758065    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.758219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.758196    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.758284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.758197    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.760234 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.760195    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e267af0e-0894-4514-815d-3bed98989354-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.760427 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.760405    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e267af0e-0894-4514-815d-3bed98989354-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.814115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.814029    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvvs\" (UniqueName: \"kubernetes.io/projected/e267af0e-0894-4514-815d-3bed98989354-kube-api-access-6tvvs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-g2h6s\" (UID: \"e267af0e-0894-4514-815d-3bed98989354\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:37.834916 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:37.834878    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:38.204437 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:38.204407    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"]
Nov 02 21:43:38.206883 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:43:38.206852    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode267af0e_0894_4514_815d_3bed98989354.slice/crio-48b918e91cb5634d48565bae2dd9106ee26b428bdc4ef4cacf94b677cbfb0893 WatchSource:0}: Error finding container 48b918e91cb5634d48565bae2dd9106ee26b428bdc4ef4cacf94b677cbfb0893: Status 404 returned error can't find the container with id 48b918e91cb5634d48565bae2dd9106ee26b428bdc4ef4cacf94b677cbfb0893
Nov 02 21:43:38.771519 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:38.771474    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" event={"ID":"e267af0e-0894-4514-815d-3bed98989354","Type":"ContainerStarted","Data":"48b918e91cb5634d48565bae2dd9106ee26b428bdc4ef4cacf94b677cbfb0893"}
Nov 02 21:43:39.777469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:39.777423    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" event={"ID":"e267af0e-0894-4514-815d-3bed98989354","Type":"ContainerStarted","Data":"24f08f597cc927078dae313f48a12387028342048a6a10e4691987b0157a2b7f"}
Nov 02 21:43:39.835620 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:39.835582    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:39.890525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:39.890469    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" podStartSLOduration=2.274003483 podStartE2EDuration="2.890451629s" totalImagesPullingTime="616.448146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:39.890252499 +0000 UTC m=+518.406748273" watchObservedRunningTime="2026-11-02 21:43:39.890451629 +0000 UTC m=+518.406947402"
Nov 02 21:43:39.926020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:39.925978    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:39.938445 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:39.938417    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:40.793610 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:40.793578    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:43:45.272136 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.272095    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"]
Nov 02 21:43:45.275904 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.275883    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.331913 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.331877    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"]
Nov 02 21:43:45.354178 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.354146    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 02 21:43:45.433089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433043    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.433089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433091    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.433304 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433240    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.433339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433318    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.433377 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433342    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108043cb-e370-4bb2-961e-191c3414ab62-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.433409 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.433400    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49lh\" (UniqueName: \"kubernetes.io/projected/108043cb-e370-4bb2-961e-191c3414ab62-kube-api-access-l49lh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534238    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534291    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534316    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108043cb-e370-4bb2-961e-191c3414ab62-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534562 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534343    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l49lh\" (UniqueName: \"kubernetes.io/projected/108043cb-e370-4bb2-961e-191c3414ab62-kube-api-access-l49lh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534562 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534380    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534562 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534414    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534731    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534806 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534759    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.534844 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.534813    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.536671 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.536633    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/108043cb-e370-4bb2-961e-191c3414ab62-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.537030 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.536971    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/108043cb-e370-4bb2-961e-191c3414ab62-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.590856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.590825    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49lh\" (UniqueName: \"kubernetes.io/projected/108043cb-e370-4bb2-961e-191c3414ab62-kube-api-access-l49lh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-gvpnj\" (UID: \"108043cb-e370-4bb2-961e-191c3414ab62\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:45.775522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.775483    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:43:45.888287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:45.888193    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:46.257172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:46.257144    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"]
Nov 02 21:43:46.259751 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:43:46.259723    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108043cb_e370_4bb2_961e_191c3414ab62.slice/crio-51cd10d88aec5774b2f511b7ebb08c2f92d73e6458ca37bc941f47e519585bdc WatchSource:0}: Error finding container 51cd10d88aec5774b2f511b7ebb08c2f92d73e6458ca37bc941f47e519585bdc: Status 404 returned error can't find the container with id 51cd10d88aec5774b2f511b7ebb08c2f92d73e6458ca37bc941f47e519585bdc
Nov 02 21:43:46.812371 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:46.812335    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj" event={"ID":"108043cb-e370-4bb2-961e-191c3414ab62","Type":"ContainerStarted","Data":"51cd10d88aec5774b2f511b7ebb08c2f92d73e6458ca37bc941f47e519585bdc"}
Nov 02 21:43:47.818621 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:47.818590    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj" event={"ID":"108043cb-e370-4bb2-961e-191c3414ab62","Type":"ContainerStarted","Data":"05c5644b3cb7b036d3642dbe71f31aa06b9e1e52cbaf030fdb7d46cd2e396bd4"}
Nov 02 21:43:47.836032 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:47.835992    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:47.848517 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:47.848486    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:47.888851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:47.888815    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:47.938607 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:47.934188    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj" podStartSLOduration=2.351242877 podStartE2EDuration="2.934169475s" totalImagesPullingTime="582.926598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:47.932617294 +0000 UTC m=+526.449113067" watchObservedRunningTime="2026-11-02 21:43:47.934169475 +0000 UTC m=+526.450665249"
Nov 02 21:43:48.834218 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:48.834186    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:43:52.862754 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:52.862718    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4txnq"]
Nov 02 21:43:52.866430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:52.866404    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:52.967853 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:52.967812    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 02 21:43:52.969296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:52.969269    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4txnq"]
Nov 02 21:43:53.003571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003530    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-home\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.003757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003586    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.003757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003705    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb30c5fd-fa38-4323-ad59-8888963b7224-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.003757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003745    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.003892 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003772    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.003892 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.003880    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtddg\" (UniqueName: \"kubernetes.io/projected/bb30c5fd-fa38-4323-ad59-8888963b7224-kube-api-access-dtddg\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.104901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.104857    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-home\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.104901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.104915    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105051    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb30c5fd-fa38-4323-ad59-8888963b7224-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105104    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105144    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105185 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105181    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtddg\" (UniqueName: \"kubernetes.io/projected/bb30c5fd-fa38-4323-ad59-8888963b7224-kube-api-access-dtddg\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105356 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105310    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-home\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105392    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.105457 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.105410    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.107361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.107331    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb30c5fd-fa38-4323-ad59-8888963b7224-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.107570 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.107552    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb30c5fd-fa38-4323-ad59-8888963b7224-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.161640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.161552    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtddg\" (UniqueName: \"kubernetes.io/projected/bb30c5fd-fa38-4323-ad59-8888963b7224-kube-api-access-dtddg\") pod \"echo-simulator-kserve-64b6f8f96c-4txnq\" (UID: \"bb30c5fd-fa38-4323-ad59-8888963b7224\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.178796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.178756    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:53.551313 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.551282    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4txnq"]
Nov 02 21:43:53.554029 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:43:53.553994    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb30c5fd_fa38_4323_ad59_8888963b7224.slice/crio-91c0f6f310e45cc7d4ac2792e318d3786ffa597214932d02eb0e49ddfa7db724 WatchSource:0}: Error finding container 91c0f6f310e45cc7d4ac2792e318d3786ffa597214932d02eb0e49ddfa7db724: Status 404 returned error can't find the container with id 91c0f6f310e45cc7d4ac2792e318d3786ffa597214932d02eb0e49ddfa7db724
Nov 02 21:43:53.851530 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:53.851449    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq" event={"ID":"bb30c5fd-fa38-4323-ad59-8888963b7224","Type":"ContainerStarted","Data":"91c0f6f310e45cc7d4ac2792e318d3786ffa597214932d02eb0e49ddfa7db724"}
Nov 02 21:43:55.888741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:55.888705    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:55.901046 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:55.901018    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:56.321577 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:43:56.321445    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:43:56.868160 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:56.868119    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq" event={"ID":"bb30c5fd-fa38-4323-ad59-8888963b7224","Type":"ContainerStarted","Data":"a89d8c34d749b2c93565e9fa7044dca4b8c0267d682f92fe49242af50be63aad"}
Nov 02 21:43:56.879337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:56.879312    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj"
Nov 02 21:43:56.982529 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:56.982469    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq" podStartSLOduration=2.127027215 podStartE2EDuration="4.982453403s" totalImagesPullingTime="2.855426188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:43:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:43:56.981089531 +0000 UTC m=+535.497585303" watchObservedRunningTime="2026-11-02 21:43:56.982453403 +0000 UTC m=+535.498949216"
Nov 02 21:43:57.179822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:57.179793    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:43:59.423793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:43:59.423757    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:44:03.179556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.179509    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:44:03.191638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.191599    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:44:03.606790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.606714    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:44:03.611537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.611514    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.621066 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.621038    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:44:03.624374 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.624353    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.663725 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.663687    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qzh4z\""
Nov 02 21:44:03.709429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709388    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.709623 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709443    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.709623 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709576    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709678    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq9f6\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709726    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709780    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709830    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.709915    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710002    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710031    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710086    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710112    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710167    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710423    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710494    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pw9p\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710581    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.710796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710626    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.710796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.710653    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.712161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.712139    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:44:03.761744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.761699    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:44:03.811261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811219    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811272    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq9f6\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811303    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811337    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811357    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811381    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811403    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811432    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811462    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811489    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811524    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811555    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811610    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pw9p\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811638    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811669    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811706    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811733    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811746    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.811809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811748    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.812395 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.811893    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.812395 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812239    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.812395 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812254    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.812552 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812437    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.812552 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812478    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.812659 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812575    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.812895 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.812873    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.813075 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.813054    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.813192 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.813166    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.814027 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.814008    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.814147 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.814127    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.814631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.814615    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.814821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.814803    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.868220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.868140    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq9f6\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.868892 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.868859    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-86qzd\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.869041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.869007    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.869114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.869048    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pw9p\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\") pod \"maas-default-gateway-openshift-default-65c448b547-kkqtt\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:03.914215 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.914175    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq"
Nov 02 21:44:03.924549 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.924508    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:03.937179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:03.937157    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:04.351317 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.351266    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:44:04.353323 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:44:04.353297    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcd8d5e_dfe5_41c9_a621_bde78828ce84.slice/crio-696ff23e6cefbcf4a3e6291434d00035f0be69c90558fb27d7dc076989ce69f7 WatchSource:0}: Error finding container 696ff23e6cefbcf4a3e6291434d00035f0be69c90558fb27d7dc076989ce69f7: Status 404 returned error can't find the container with id 696ff23e6cefbcf4a3e6291434d00035f0be69c90558fb27d7dc076989ce69f7
Nov 02 21:44:04.354007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.353941    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:44:04.355394 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:44:04.355371    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61b4e2f_b51b_48e6_9d7d_ead80679ce36.slice/crio-0775864636ab322dda48d4c13bfd3b25c8697706cee805ecb47e263983463b1f WatchSource:0}: Error finding container 0775864636ab322dda48d4c13bfd3b25c8697706cee805ecb47e263983463b1f: Status 404 returned error can't find the container with id 0775864636ab322dda48d4c13bfd3b25c8697706cee805ecb47e263983463b1f
Nov 02 21:44:04.355830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.355791    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.355925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.355909    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.355988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.355969    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.357809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.357755    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.357850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.357822    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.357883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.357868    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:44:04.937079 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.937048    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" event={"ID":"6dcd8d5e-dfe5-41c9-a621-bde78828ce84","Type":"ContainerStarted","Data":"858ce37746ca757f67ecce6f1fa62de2de8a1aa3cb12f4d5d147186a8182fcd3"}
Nov 02 21:44:04.937296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.937103    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" event={"ID":"6dcd8d5e-dfe5-41c9-a621-bde78828ce84","Type":"ContainerStarted","Data":"696ff23e6cefbcf4a3e6291434d00035f0be69c90558fb27d7dc076989ce69f7"}
Nov 02 21:44:04.938857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.938826    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" event={"ID":"e61b4e2f-b51b-48e6-9d7d-ead80679ce36","Type":"ContainerStarted","Data":"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"}
Nov 02 21:44:04.939003 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:04.938873    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" event={"ID":"e61b4e2f-b51b-48e6-9d7d-ead80679ce36","Type":"ContainerStarted","Data":"0775864636ab322dda48d4c13bfd3b25c8697706cee805ecb47e263983463b1f"}
Nov 02 21:44:05.061044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.060987    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podStartSLOduration=2.060973531 podStartE2EDuration="2.060973531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:44:05.06035391 +0000 UTC m=+543.576849684" watchObservedRunningTime="2026-11-02 21:44:05.060973531 +0000 UTC m=+543.577469294"
Nov 02 21:44:05.179981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.177519    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podStartSLOduration=2.177499628 podStartE2EDuration="2.177499628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:44:05.174748737 +0000 UTC m=+543.691244506" watchObservedRunningTime="2026-11-02 21:44:05.177499628 +0000 UTC m=+543.693995401"
Nov 02 21:44:05.929998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.929351    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:05.929998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.929420    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:05.929998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.929361    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:05.938108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.938076    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:05.939316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.939288    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:05.939452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:05.939349    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:06.928423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:06.928379    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:06.928636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:06.928465    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:06.937790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:06.937750    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:06.938197 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:06.937804    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:07.934065 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:07.934030    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:07.934226 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:07.934096    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:07.938456 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:07.938423    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:07.938815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:07.938487    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:08.933039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:08.932114    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:08.933039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:08.932188    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:08.938272 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:08.938237    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:08.938447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:08.938285    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:09.927387 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:09.927348    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:09.928041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:09.928004    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:09.938473 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:09.938432    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:09.938784 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:09.938754    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:10.925751 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:10.925710    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 02 21:44:10.925932 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:10.925782    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 02 21:44:10.938217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:10.938177    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kkqtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 02 21:44:10.938560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:10.938250    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 02 21:44:12.575801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.575710    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:12.576583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.576522    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:12.577417 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.577394    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:44:12.644520 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.644479    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:12.644884 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.644853    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:12.645755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:44:12.645737    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:44:26.311039 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:44:26.311001    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:44:56.311311 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:44:56.311270    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:45:01.655773 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:01.655742    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:45:01.656491 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:01.656472    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:45:01.672900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:01.672871    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:45:01.673931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:01.673904    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:45:04.083930 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.083887    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:45:04.088093 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.088061    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.133777 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.133742    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:45:04.137269 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.137249    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.216991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.216926    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.216991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.216986    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217066    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217097    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ghnf\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217128    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217229    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217295    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217344    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217365    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217385    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217402    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217467    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217487    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pljbk\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217849 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217509    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217849 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217526    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217849 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217544    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.217849 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217612    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.217849 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.217648    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.231146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.231108    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:45:04.288129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.288090    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:45:04.319090 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319050    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319114    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319135    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319157    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319278    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ghnf\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319337    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319386    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319463    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319503    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319532    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319542    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319559    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319583    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319624    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319622    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319653    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pljbk\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319691    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319718    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319726    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319741    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319792    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319812    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.319903 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.319902    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.320426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.320076    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.320426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.320137    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.320426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.320290    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.320662 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.320636    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.320731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.320665    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.322081 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.321789    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.322601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.322578    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.322738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.322699    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.322738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.322713    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.375698 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.375624    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.375914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.375889    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pljbk\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\") pod \"maas-default-gateway-openshift-default-65c448b547-5qjbw\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.375988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.375895    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ghnf\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.376203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.376185    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xx4ks\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.402635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.402598    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:04.450187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.450141    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:04.830835 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.830566    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:45:04.833564 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:04.833538    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f628df_175c_42e9_ac99_682bb262cc59.slice/crio-6698eb4a98b75e2281e60aac5f27ff3fd43084195b602d0b92992a845ce6aacc WatchSource:0}: Error finding container 6698eb4a98b75e2281e60aac5f27ff3fd43084195b602d0b92992a845ce6aacc: Status 404 returned error can't find the container with id 6698eb4a98b75e2281e60aac5f27ff3fd43084195b602d0b92992a845ce6aacc
Nov 02 21:45:04.834480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.834420    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:45:04.835696 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:04.835674    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d5ec43_1709_43d7_97b1_57649fa5c2de.slice/crio-474c880aa79998cf508c1d0e827962da44ec6544f00ed692fe6f8470ec3201b8 WatchSource:0}: Error finding container 474c880aa79998cf508c1d0e827962da44ec6544f00ed692fe6f8470ec3201b8: Status 404 returned error can't find the container with id 474c880aa79998cf508c1d0e827962da44ec6544f00ed692fe6f8470ec3201b8
Nov 02 21:45:04.836453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.836366    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:04.836502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.836448    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:04.836502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.836484    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:04.838099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.838073    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:04.838201 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.838133    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:04.838201 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:04.838168    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:05.245841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.245795    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" event={"ID":"c3d5ec43-1709-43d7-97b1-57649fa5c2de","Type":"ContainerStarted","Data":"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"}
Nov 02 21:45:05.246333 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.245864    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" event={"ID":"c3d5ec43-1709-43d7-97b1-57649fa5c2de","Type":"ContainerStarted","Data":"474c880aa79998cf508c1d0e827962da44ec6544f00ed692fe6f8470ec3201b8"}
Nov 02 21:45:05.247553 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.247528    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" event={"ID":"e5f628df-175c-42e9-ac99-682bb262cc59","Type":"ContainerStarted","Data":"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"}
Nov 02 21:45:05.247703 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.247567    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" event={"ID":"e5f628df-175c-42e9-ac99-682bb262cc59","Type":"ContainerStarted","Data":"6698eb4a98b75e2281e60aac5f27ff3fd43084195b602d0b92992a845ce6aacc"}
Nov 02 21:45:05.361644 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.361589    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podStartSLOduration=1.3615756700000001 podStartE2EDuration="1.36157567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:05.36055583 +0000 UTC m=+603.877051608" watchObservedRunningTime="2026-11-02 21:45:05.36157567 +0000 UTC m=+603.878071439"
Nov 02 21:45:05.406982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.406826    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:05.407166 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.406992    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:05.407166 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.407039    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:05.450610 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.450570    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:05.451681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.451656    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:05.451779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.451695    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:05.484001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:05.482769    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podStartSLOduration=1.482747392 podStartE2EDuration="1.482747392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:05.480377167 +0000 UTC m=+603.996872953" watchObservedRunningTime="2026-11-02 21:45:05.482747392 +0000 UTC m=+603.999243170"
Nov 02 21:45:06.407873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:06.407364    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:06.407873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:06.407456    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:06.451471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:06.451435    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:06.451677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:06.451495    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:07.403468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:07.403425    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:07.403761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:07.403727    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:07.451362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:07.451326    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:07.451756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:07.451384    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:08.413982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:08.411379    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:08.413982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:08.411444    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:08.450808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:08.450763    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:08.451012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:08.450822    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:09.403478 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:09.403432    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:09.404146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:09.404108    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:09.451347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:09.451311    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:09.451583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:09.451557    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:10.403591 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:10.403540    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:10.404374 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:10.404311    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:10.451523 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:10.451486    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:10.451699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:10.451542    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:11.403673 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:11.403629    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5qjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 02 21:45:11.404221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:11.403689    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 02 21:45:11.451247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:11.451200    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xx4ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 02 21:45:11.451426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:11.451278    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 02 21:45:13.108938 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.108902    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:13.110024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.110001    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:13.110584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.110566    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:45:13.133078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.133041    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:13.133449 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.133420    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:13.134397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:13.134374    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:45:22.823020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.822986    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 21:45:22.826815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.826785    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875389 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875350    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875397    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875494    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875557    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zlzt\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875600    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875643    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875821    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.875923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875907    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.876007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.875940    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.878640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.878614    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-9cc56b-openshift-default-dockercfg-hl779\""
Nov 02 21:45:22.930040 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.930002    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 21:45:22.977231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977185    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977232    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977287    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977322    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977360    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977389    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlzt\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977420    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977461    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977578    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977711    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977755    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.977785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.977803    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.978088 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.978008    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.978346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.978324    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.979756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.979734    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:22.979821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:22.979802    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:23.039596 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.039508    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:23.039772 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.039653    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlzt\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:23.121931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.121843    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 21:45:23.126869 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.126843    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.140841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.140808    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:23.178991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.178945    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.179339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.179318    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.179467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.179454    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.179582 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.179570    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.179681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.179667    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhpnl\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.179790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.179777    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.180042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.180025    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.180177 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.180164    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.180298 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.180287    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.180708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.180685    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f48c4d-openshift-default-dockercfg-xpv97\""
Nov 02 21:45:23.213341 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.213301    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 21:45:23.281175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281137    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281252    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281293    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281320    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281344    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhpnl\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281380    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281437    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281467    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281516    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281612    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281884 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281862    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.281984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.281937    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.282568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.282182    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.286415 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.286387    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.286645 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.286623    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.288946 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.288911    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.362007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.361074    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.362007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.361197    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhpnl\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.441837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.441795    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:23.608779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.608753    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 21:45:23.611188 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:23.611154    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d111d11_ff27_423b_b38c_2e522b0ae4d3.slice/crio-ca069f39f107e66c463b2a9b854c5762ffc31f4427a1a1255d55002c0d3f5986 WatchSource:0}: Error finding container ca069f39f107e66c463b2a9b854c5762ffc31f4427a1a1255d55002c0d3f5986: Status 404 returned error can't find the container with id ca069f39f107e66c463b2a9b854c5762ffc31f4427a1a1255d55002c0d3f5986
Nov 02 21:45:23.613782 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.613735    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:23.613919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.613837    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:23.613919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.613890    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:23.819358 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.819291    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 21:45:23.824478 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:23.824279    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d26d6f_037d_4880_ad82_5ac26e61ff20.slice/crio-a1c86f0ac23c3f14eaff94da2216877cdbe50243cb826c8b1be511cd6571d6fe WatchSource:0}: Error finding container a1c86f0ac23c3f14eaff94da2216877cdbe50243cb826c8b1be511cd6571d6fe: Status 404 returned error can't find the container with id a1c86f0ac23c3f14eaff94da2216877cdbe50243cb826c8b1be511cd6571d6fe
Nov 02 21:45:23.827656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.827620    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:23.827753 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.827735    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:23.827803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:23.827783    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:24.105913 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.105840    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 21:45:24.109669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.109645    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.170447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.162130    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-96c052-openshift-default-dockercfg-fh882\""
Nov 02 21:45:24.170447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.168731    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 21:45:24.197047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.196818    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.196890    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.196910    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.196981    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.197009    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.197094    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.197198    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.197231    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.197430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.197257    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc7lb\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.298901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.298852    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.298901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.298895    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.298914    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.298941    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.298998    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299018    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc7lb\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299107    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299170    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299193    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299536 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299382    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299536 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299425    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299582    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299690 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299665    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.299878 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.299857    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.301328 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.301303    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.301564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.301546    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.355388 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.355351    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.355532 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.355459    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc7lb\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.375909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.375823    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerStarted","Data":"b1a0f5515b9421d48add551c103edea95cb248f6ae25dc961542b97073852340"}
Nov 02 21:45:24.375909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.375885    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerStarted","Data":"ca069f39f107e66c463b2a9b854c5762ffc31f4427a1a1255d55002c0d3f5986"}
Nov 02 21:45:24.377508 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.377483    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerStarted","Data":"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"}
Nov 02 21:45:24.377628 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.377524    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerStarted","Data":"a1c86f0ac23c3f14eaff94da2216877cdbe50243cb826c8b1be511cd6571d6fe"}
Nov 02 21:45:24.423120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.423085    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:24.442601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.442562    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:24.458396 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.458364    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:24.497026 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.495883    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podStartSLOduration=2.495862747 podStartE2EDuration="2.495862747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:24.495778982 +0000 UTC m=+623.012274756" watchObservedRunningTime="2026-11-02 21:45:24.495862747 +0000 UTC m=+623.012358520"
Nov 02 21:45:24.615381 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.614693    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podStartSLOduration=1.614672605 podStartE2EDuration="1.614672605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:24.613436823 +0000 UTC m=+623.129932595" watchObservedRunningTime="2026-11-02 21:45:24.614672605 +0000 UTC m=+623.131168380"
Nov 02 21:45:24.859763 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.859702    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 21:45:24.862127 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:24.862097    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f720ff_0515_4447_a170_ddf3d263d6cf.slice/crio-3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922 WatchSource:0}: Error finding container 3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922: Status 404 returned error can't find the container with id 3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922
Nov 02 21:45:24.864615 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.864580    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:24.864699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.864651    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:24.864699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:24.864682    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:45:25.142035 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.141908    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:25.148168 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.148141    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:25.384465 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.384424    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerStarted","Data":"3abfec66a24db755329be418a0294a3b1c5b9e8170e63afbd682571ec1579b6f"}
Nov 02 21:45:25.384662 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.384487    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerStarted","Data":"3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922"}
Nov 02 21:45:25.384909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.384881    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:25.385038 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.384936    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:25.385680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.385651    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:45:25.385771 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.385714    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:45:25.423944 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.423913    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:25.429229 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.429201    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:25.503238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:25.503183    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podStartSLOduration=1.503169049 podStartE2EDuration="1.503169049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:25.501037337 +0000 UTC m=+624.017533109" watchObservedRunningTime="2026-11-02 21:45:25.503169049 +0000 UTC m=+624.019664821"
Nov 02 21:45:26.312549 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:26.312501    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:45:26.390179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:26.390132    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:26.390943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:26.390920    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:45:33.340471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.340435    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 21:45:33.344847 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.344818    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.406693 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.406644    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.406851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.406720    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.406905 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.406840    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn74r\" (UniqueName: \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.407008 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.406988    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.447429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.447396    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-fb8112-serving-cert\""
Nov 02 21:45:33.449382 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.449359    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 21:45:33.508301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.508251    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.508509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.508398    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.508509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.508443    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.508509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.508475    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn74r\" (UniqueName: \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.508670 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:33.508581    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-serving-cert: secret "maas-api-e2e-worker-w6-fb8112-serving-cert" not found
Nov 02 21:45:33.508670 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:33.508666    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls podName:91a8e487-5b23-4a12-8cda-0c8e8988779d nodeName:}" failed. No retries permitted until 2026-11-02 21:45:34.008643463 +0000 UTC m=+632.525139241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls") pod "maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" (UID: "91a8e487-5b23-4a12-8cda-0c8e8988779d") : secret "maas-api-e2e-worker-w6-fb8112-serving-cert" not found
Nov 02 21:45:33.509672 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.509646    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.511369 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.511352    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:33.565316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:33.565274    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn74r\" (UniqueName: \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:34.014483 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:34.014442    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:34.016965 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:34.016927    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") pod \"maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:34.258507 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:34.258461    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:34.629088 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:34.628994    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 21:45:34.631945 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:34.631909    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a8e487_5b23_4a12_8cda_0c8e8988779d.slice/crio-b732477b29309c34576b5a27b45c5bcaa2b6675ec8c1d84840896b4953c1cc3f WatchSource:0}: Error finding container b732477b29309c34576b5a27b45c5bcaa2b6675ec8c1d84840896b4953c1cc3f: Status 404 returned error can't find the container with id b732477b29309c34576b5a27b45c5bcaa2b6675ec8c1d84840896b4953c1cc3f
Nov 02 21:45:35.443435 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:35.443374    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" event={"ID":"91a8e487-5b23-4a12-8cda-0c8e8988779d","Type":"ContainerStarted","Data":"b732477b29309c34576b5a27b45c5bcaa2b6675ec8c1d84840896b4953c1cc3f"}
Nov 02 21:45:36.450289 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:36.450255    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" event={"ID":"91a8e487-5b23-4a12-8cda-0c8e8988779d","Type":"ContainerStarted","Data":"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"}
Nov 02 21:45:36.450765 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:36.450455    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:36.565089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:36.565029    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" podStartSLOduration=2.734312867 podStartE2EDuration="3.565014131s" totalImagesPullingTime="830.701264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:36.563320344 +0000 UTC m=+635.079816116" watchObservedRunningTime="2026-11-02 21:45:36.565014131 +0000 UTC m=+635.081509902"
Nov 02 21:45:41.618703 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:41.618667    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:45:42.461275 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:42.461244    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 21:45:43.879586 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:43.879541    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 21:45:43.884019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:43.883992    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:43.936207 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:43.936175    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-f48c4d-serving-cert\""
Nov 02 21:45:43.999140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:43.999099    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 21:45:44.023408 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.023377    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.023997 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.023696    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.024315 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.024293    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.024638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.024615    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8cqd\" (UniqueName: \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.126642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.126598    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8cqd\" (UniqueName: \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.126822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.126778    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.126895 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.126880    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.126943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.126925    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.127078 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:44.127055    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-serving-cert: secret "maas-api-e2e-worker-w0-f48c4d-serving-cert" not found
Nov 02 21:45:44.127164 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:44.127151    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls podName:ba18e99d-dd3b-4176-a6ed-741b645187dd nodeName:}" failed. No retries permitted until 2026-11-02 21:45:44.627122111 +0000 UTC m=+643.143617882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls") pod "maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" (UID: "ba18e99d-dd3b-4176-a6ed-741b645187dd") : secret "maas-api-e2e-worker-w0-f48c4d-serving-cert" not found
Nov 02 21:45:44.127768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.127741    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.129494 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.129465    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.184354 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.184323    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8cqd\" (UniqueName: \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.632584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.632474    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.634966 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.634921    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:44.797086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:44.797051    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:45.170891 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:45.170866    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 21:45:45.173341 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:45.173311    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba18e99d_dd3b_4176_a6ed_741b645187dd.slice/crio-7c239d0542b7c130b64ba4603f09d445b0c9e518e883961c732e77470b9398bc WatchSource:0}: Error finding container 7c239d0542b7c130b64ba4603f09d445b0c9e518e883961c732e77470b9398bc: Status 404 returned error can't find the container with id 7c239d0542b7c130b64ba4603f09d445b0c9e518e883961c732e77470b9398bc
Nov 02 21:45:45.505469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:45.505371    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" event={"ID":"ba18e99d-dd3b-4176-a6ed-741b645187dd","Type":"ContainerStarted","Data":"7c239d0542b7c130b64ba4603f09d445b0c9e518e883961c732e77470b9398bc"}
Nov 02 21:45:46.527356 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:46.527315    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" event={"ID":"ba18e99d-dd3b-4176-a6ed-741b645187dd","Type":"ContainerStarted","Data":"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"}
Nov 02 21:45:46.527790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:46.527466    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:46.641871 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:46.641821    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" podStartSLOduration=2.799670154 podStartE2EDuration="3.641805641s" totalImagesPullingTime="842.135487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:46.6409785 +0000 UTC m=+645.157474274" watchObservedRunningTime="2026-11-02 21:45:46.641805641 +0000 UTC m=+645.158301413"
Nov 02 21:45:50.731459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:50.731420    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:45:52.538277 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:52.538252    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 21:45:54.030993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.030925    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 21:45:54.035112 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.035089    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.152142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.152103    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-96c052-serving-cert\""
Nov 02 21:45:54.153126 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.153097    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 21:45:54.170290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.170256    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.170451 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.170301    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.170451 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.170401    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzmqw\" (UniqueName: \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.170451 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.170432    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.271575 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.271538    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.271757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.271584    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.271757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.271631    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmqw\" (UniqueName: \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.271757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.271657    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.271757 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:54.271695    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-serving-cert: secret "maas-api-e2e-worker-w5-96c052-serving-cert" not found
Nov 02 21:45:54.271892 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:54.271774    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls podName:65528f8c-552e-47dc-af4c-6ac6c8d5324f nodeName:}" failed. No retries permitted until 2026-11-02 21:45:54.771757281 +0000 UTC m=+653.288253057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls") pod "maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" (UID: "65528f8c-552e-47dc-af4c-6ac6c8d5324f") : secret "maas-api-e2e-worker-w5-96c052-serving-cert" not found
Nov 02 21:45:54.272330 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.272298    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.274482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.274460    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.346363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.346288    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzmqw\" (UniqueName: \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.793318 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.792736    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.810572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.810538    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") pod \"maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:54.948387 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:54.948346    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:55.370775 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:55.370750    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 21:45:55.373375 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:45:55.373337    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65528f8c_552e_47dc_af4c_6ac6c8d5324f.slice/crio-bbb965229ced19341a1801fb7d976181422e5b634f3eec6b4c9f611a02c384b4 WatchSource:0}: Error finding container bbb965229ced19341a1801fb7d976181422e5b634f3eec6b4c9f611a02c384b4: Status 404 returned error can't find the container with id bbb965229ced19341a1801fb7d976181422e5b634f3eec6b4c9f611a02c384b4
Nov 02 21:45:55.626044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:55.625508    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" event={"ID":"65528f8c-552e-47dc-af4c-6ac6c8d5324f","Type":"ContainerStarted","Data":"bbb965229ced19341a1801fb7d976181422e5b634f3eec6b4c9f611a02c384b4"}
Nov 02 21:45:56.360870 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:45:56.360825    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:45:56.635764 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:56.635669    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" event={"ID":"65528f8c-552e-47dc-af4c-6ac6c8d5324f","Type":"ContainerStarted","Data":"490407b28aa6e27b9170308b5010eb451eb1f978b5d2a2bb8ad8ffd4fe5e485b"}
Nov 02 21:45:56.636246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:56.635898    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:45:56.772702 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:45:56.772643    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" podStartSLOduration=2.933500488 podStartE2EDuration="3.772625606s" totalImagesPullingTime="839.125118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:45:56.771161329 +0000 UTC m=+655.287657103" watchObservedRunningTime="2026-11-02 21:45:56.772625606 +0000 UTC m=+655.289121435"
Nov 02 21:46:00.660795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.660759    5580 generic.go:356] "Generic (PLEG): container finished" podID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerID="b1a0f5515b9421d48add551c103edea95cb248f6ae25dc961542b97073852340" exitCode=0
Nov 02 21:46:00.661267 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.660829    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerDied","Data":"b1a0f5515b9421d48add551c103edea95cb248f6ae25dc961542b97073852340"}
Nov 02 21:46:00.661332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.661268    5580 scope.go:122] "RemoveContainer" containerID="b1a0f5515b9421d48add551c103edea95cb248f6ae25dc961542b97073852340"
Nov 02 21:46:00.662623 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662602    5580 generic.go:356] "Generic (PLEG): container finished" podID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerID="4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc" exitCode=0
Nov 02 21:46:00.662708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662669    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerDied","Data":"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"}
Nov 02 21:46:00.662759 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662714    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.662794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662784    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.662838 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662818    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.663003 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.662990    5580 scope.go:122] "RemoveContainer" containerID="4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"
Nov 02 21:46:00.664247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.664218    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.664316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.664298    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.664361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.664346    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:00.942336 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:00.942290    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:46:01.670339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:01.670232    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerStarted","Data":"2a7214109965d944754327e2089097f5768cf80d539a7d8d13b8914c387ebd12"}
Nov 02 21:46:01.672216 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:01.672189    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerStarted","Data":"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"}
Nov 02 21:46:02.142047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.141378    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:46:02.142047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.141502    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 02 21:46:02.142047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.141544    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 02 21:46:02.444444 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.444406    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:46:02.444837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.444809    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 02 21:46:02.445041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.445014    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 02 21:46:02.653572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:02.653539    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 21:46:03.150982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:03.141394    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 02 21:46:03.150982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:03.141477    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 02 21:46:03.444984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:03.444407    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 02 21:46:03.444984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:03.444480    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 02 21:46:04.148442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:04.148399    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 02 21:46:04.148748 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:04.148718    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 02 21:46:04.442653 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:04.442610    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 02 21:46:04.443305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:04.443271    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 02 21:46:05.150471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:05.150426    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 02 21:46:05.150786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:05.150753    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 02 21:46:05.444984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:05.444429    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 02 21:46:05.444984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:05.444497    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 02 21:46:06.141516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:06.141481    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 02 21:46:06.141821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:06.141792    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 02 21:46:06.444980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:06.444774    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 02 21:46:06.444980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:06.444852    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 02 21:46:07.145883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.145849    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:46:07.146298 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.146274    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:46:07.146982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.146949    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 21:46:07.456335 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.456296    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:46:07.457161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.457090    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:46:07.458475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:07.458447    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 21:46:10.733488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.733449    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:46:10.811666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.811632    5580 generic.go:356] "Generic (PLEG): container finished" podID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerID="3abfec66a24db755329be418a0294a3b1c5b9e8170e63afbd682571ec1579b6f" exitCode=0
Nov 02 21:46:10.811836 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.811709    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerDied","Data":"3abfec66a24db755329be418a0294a3b1c5b9e8170e63afbd682571ec1579b6f"}
Nov 02 21:46:10.812142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.812124    5580 scope.go:122] "RemoveContainer" containerID="3abfec66a24db755329be418a0294a3b1c5b9e8170e63afbd682571ec1579b6f"
Nov 02 21:46:10.813577 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.813550    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:10.813635 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.813626    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:10.813668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:10.813661    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:46:11.393985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:11.392076    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:46:11.818515 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:11.818467    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerStarted","Data":"325e5e8bdbb5285b2e00ce0cd849d7f32629eda6d2880f44c1a334aa46e4e82f"}
Nov 02 21:46:12.423940 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:12.423906    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:46:12.425001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:12.424951    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 02 21:46:12.425143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:12.425031    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 02 21:46:13.424074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.424023    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 02 21:46:13.424541 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.424087    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 02 21:46:13.770137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.770051    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 21:46:13.772921 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.772895    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.833504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.830703    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-ffe386-serving-cert\""
Nov 02 21:46:13.833697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.833582    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 21:46:13.842492 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.842456    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfww9\" (UniqueName: \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.842676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.842593    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.842676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.842610    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.842809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.842679    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.943716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.943679    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.943716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.943718    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.944038 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:46:13.943832    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-serving-cert: secret "maas-api-e2e-worker-w1-ffe386-serving-cert" not found
Nov 02 21:46:13.944038 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:46:13.943918    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls podName:0353af85-1789-481a-8933-8d40027ceff9 nodeName:}" failed. No retries permitted until 2026-11-02 21:46:14.443894832 +0000 UTC m=+672.960390589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls") pod "maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" (UID: "0353af85-1789-481a-8933-8d40027ceff9") : secret "maas-api-e2e-worker-w1-ffe386-serving-cert" not found
Nov 02 21:46:13.944038 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.943840    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.944184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.944059    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfww9\" (UniqueName: \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.944582 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.944561    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:13.946248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:13.946230    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:14.003397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.003355    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfww9\" (UniqueName: \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:14.423420 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.423384    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 02 21:46:14.423594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.423437    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 02 21:46:14.449710 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.449672    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:14.452358 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.452332    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") pod \"maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:14.715544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:14.715456    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:15.094202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:15.094098    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 21:46:15.105872 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:46:15.105836    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0353af85_1789_481a_8933_8d40027ceff9.slice/crio-08629525c34ff04e95b0275a4f0aed0d79e6f566ff4454f7945c61ef12d94f8c WatchSource:0}: Error finding container 08629525c34ff04e95b0275a4f0aed0d79e6f566ff4454f7945c61ef12d94f8c: Status 404 returned error can't find the container with id 08629525c34ff04e95b0275a4f0aed0d79e6f566ff4454f7945c61ef12d94f8c
Nov 02 21:46:15.423655 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:15.423620    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 02 21:46:15.423857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:15.423688    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 02 21:46:15.844550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:15.844454    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" event={"ID":"0353af85-1789-481a-8933-8d40027ceff9","Type":"ContainerStarted","Data":"08629525c34ff04e95b0275a4f0aed0d79e6f566ff4454f7945c61ef12d94f8c"}
Nov 02 21:46:16.423634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:16.423601    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 02 21:46:16.423809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:16.423655    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 02 21:46:16.851733 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:16.851629    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" event={"ID":"0353af85-1789-481a-8933-8d40027ceff9","Type":"ContainerStarted","Data":"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"}
Nov 02 21:46:16.851733 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:16.851724    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:16.966475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:16.966417    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" podStartSLOduration=3.157563012 podStartE2EDuration="3.966402108s" totalImagesPullingTime="808.839096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:46:16.96444629 +0000 UTC m=+675.480942064" watchObservedRunningTime="2026-11-02 21:46:16.966402108 +0000 UTC m=+675.482897881"
Nov 02 21:46:17.428990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:17.428929    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:46:17.429262 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:17.429237    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:46:17.430060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:17.430042    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 21:46:22.862808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:22.862779    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 21:46:26.311726 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:46:26.311681    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:46:29.834532 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:29.834489    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:46:52.506544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:52.506509    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:46:55.282812 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.282777    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 21:46:55.285834 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.285816    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.340504 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.340471    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"openshift-service-ca.crt\""
Nov 02 21:46:55.390202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.390157    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"kube-root-ca.crt\""
Nov 02 21:46:55.390416 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.390232    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"simulator-w0-f48c4d-kserve-self-signed-certs\""
Nov 02 21:46:55.390416 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.390296    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"default-dockercfg-d2qzt\""
Nov 02 21:46:55.391997 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.391968    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 21:46:55.417814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.417776    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.418079 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.417817    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.418079 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.417952    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.418079 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.418045    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.418246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.418157    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwgv\" (UniqueName: \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.418246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.418200    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.519833 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.519800    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.519833 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.519843    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.519941    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwgv\" (UniqueName: \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520012    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520042    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520059    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520497 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520473    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520502    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.520585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.520554    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.522973 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.522934    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.524186 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.524164    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.577270 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.577199    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwgv\" (UniqueName: \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\") pod \"simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") " pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.596625 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.596585    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:46:55.972284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:55.972259    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 21:46:55.974804 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:46:55.974771    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbf12759_76ee_4a9f_b430_1431d6d211d9.slice/crio-6309b53a9eb2d42b7ddf0bc1ab6b58832cc8ce5d462bb6b7e1a59ca9a9caa3e2 WatchSource:0}: Error finding container 6309b53a9eb2d42b7ddf0bc1ab6b58832cc8ce5d462bb6b7e1a59ca9a9caa3e2: Status 404 returned error can't find the container with id 6309b53a9eb2d42b7ddf0bc1ab6b58832cc8ce5d462bb6b7e1a59ca9a9caa3e2
Nov 02 21:46:56.082248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:56.082211    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" event={"ID":"dbf12759-76ee-4a9f-b430-1431d6d211d9","Type":"ContainerStarted","Data":"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"}
Nov 02 21:46:56.082248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:56.082255    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" event={"ID":"dbf12759-76ee-4a9f-b430-1431d6d211d9","Type":"ContainerStarted","Data":"6309b53a9eb2d42b7ddf0bc1ab6b58832cc8ce5d462bb6b7e1a59ca9a9caa3e2"}
Nov 02 21:46:56.196294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:56.196240    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podStartSLOduration=1.1962281479999999 podStartE2EDuration="1.196228148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:46:56.195704822 +0000 UTC m=+714.712200596" watchObservedRunningTime="2026-11-02 21:46:56.196228148 +0000 UTC m=+714.712723921"
Nov 02 21:46:56.314605 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:46:56.314485    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:46:56.597267 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:46:56.597182    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:47:05.597817 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:05.597764    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:47:05.610415 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:05.610383    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:47:06.145382 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.145339    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:47:06.645285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.645249    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 21:47:06.648182 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.648162    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.702430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.702393    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-96c052\"/\"openshift-service-ca.crt\""
Nov 02 21:47:06.704798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.704762    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 21:47:06.751380 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.751351    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-96c052\"/\"kube-root-ca.crt\""
Nov 02 21:47:06.758942 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.758903    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgwl\" (UniqueName: \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.759120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.758987    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.759200 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.759177    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.759261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.759226    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.759393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.759374    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.759521 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.759500    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.780614 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.780578    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-96c052\"/\"default-dockercfg-xxmgz\""
Nov 02 21:47:06.780614 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.780601    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-96c052\"/\"simulator-w5-96c052-kserve-self-signed-certs\""
Nov 02 21:47:06.861142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861098    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgwl\" (UniqueName: \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861351 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861160    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861351 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861256    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861351 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861308    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861360    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861432    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861740 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861715    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861862 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861797    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.861925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.861895    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.863616 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.863591    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.863853 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.863836    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.917477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.917401    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgwl\" (UniqueName: \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\") pod \"simulator-w5-96c052-kserve-867d8868d9-qv7nj\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") " pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:06.959384 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:06.959342    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:07.336892 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:07.336858    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 21:47:07.340126 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:47:07.340102    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5843dfcc_2d55_4553_a53e_67e0de3b4b2f.slice/crio-dabbc79aa005ee5b65d39b40d8f8a20f2ff637c3eaad66984f3f21e0e84a631f WatchSource:0}: Error finding container dabbc79aa005ee5b65d39b40d8f8a20f2ff637c3eaad66984f3f21e0e84a631f: Status 404 returned error can't find the container with id dabbc79aa005ee5b65d39b40d8f8a20f2ff637c3eaad66984f3f21e0e84a631f
Nov 02 21:47:08.148941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:08.148894    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" event={"ID":"5843dfcc-2d55-4553-a53e-67e0de3b4b2f","Type":"ContainerStarted","Data":"d5e6982298825ee89d70cc5c81c6baf8dcb7a1467a5d9dcf956a470dd51fb702"}
Nov 02 21:47:08.148941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:08.148951    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" event={"ID":"5843dfcc-2d55-4553-a53e-67e0de3b4b2f","Type":"ContainerStarted","Data":"dabbc79aa005ee5b65d39b40d8f8a20f2ff637c3eaad66984f3f21e0e84a631f"}
Nov 02 21:47:08.262342 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:08.262269    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podStartSLOduration=2.262247013 podStartE2EDuration="2.262247013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:47:08.260666854 +0000 UTC m=+726.777162626" watchObservedRunningTime="2026-11-02 21:47:08.262247013 +0000 UTC m=+726.778742787"
Nov 02 21:47:08.959751 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:08.959701    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:11.618921 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:11.618877    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:47:16.959988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:16.959916    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:16.972705 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:16.972676    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:17.218057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:17.217952    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:47:26.311489 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:47:26.311452    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:47:35.153816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:35.153779    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:47:56.310949 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:47:56.310911    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:47:59.404022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:47:59.403980    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:48:14.704490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.704425    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 21:48:14.708997 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.708951    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.729702 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729666    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqwdl\" (UniqueName: \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.729870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729715    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.729870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729808    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.729870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729860    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.730000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729921    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.730000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.729946    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.811561 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.811518    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"premium-w0-f48c4d-kserve-self-signed-certs\""
Nov 02 21:48:14.813539 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.813516    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 21:48:14.831998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.831903    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqwdl\" (UniqueName: \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832174 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832033    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832174 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832082    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832174 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832134    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832174 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832160    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832367 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832187    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832682 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832657    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832691    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.832795 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.832719    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.834451 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.834424    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.834580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.834492    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:14.888560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:14.888524    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqwdl\" (UniqueName: \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\") pod \"premium-w0-f48c4d-kserve-67dc4b45f-scpxz\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") " pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:15.021247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:15.021147    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:15.396135 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:15.396107    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 21:48:15.398163 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:48:15.398132    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f49eecd_1382_43d4_a04a_1d0765533024.slice/crio-2214c2cf5ccaf7ddcce75a0775e1868f1a2a7ea3fac2275ccf29ddb63bf167df WatchSource:0}: Error finding container 2214c2cf5ccaf7ddcce75a0775e1868f1a2a7ea3fac2275ccf29ddb63bf167df: Status 404 returned error can't find the container with id 2214c2cf5ccaf7ddcce75a0775e1868f1a2a7ea3fac2275ccf29ddb63bf167df
Nov 02 21:48:15.523265 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:15.523226    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" event={"ID":"3f49eecd-1382-43d4-a04a-1d0765533024","Type":"ContainerStarted","Data":"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"}
Nov 02 21:48:15.523428 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:15.523276    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" event={"ID":"3f49eecd-1382-43d4-a04a-1d0765533024","Type":"ContainerStarted","Data":"2214c2cf5ccaf7ddcce75a0775e1868f1a2a7ea3fac2275ccf29ddb63bf167df"}
Nov 02 21:48:15.636175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:15.636122    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podStartSLOduration=1.636106844 podStartE2EDuration="1.636106844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:48:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:48:15.634915817 +0000 UTC m=+794.151411592" watchObservedRunningTime="2026-11-02 21:48:15.636106844 +0000 UTC m=+794.152602616"
Nov 02 21:48:16.021339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:16.021305    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:22.709737 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:22.709702    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:48:25.021499 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:25.021440    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:25.034199 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:25.034167    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:25.593241 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:25.593213    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:48:26.310604 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:48:26.310566    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:48:29.228748 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.228718    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 21:48:29.232787 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.232757    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.335598 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.335560    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-96c052\"/\"premium-w5-96c052-kserve-self-signed-certs\""
Nov 02 21:48:29.337632 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.337610    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 21:48:29.409765 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.409730    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.409985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.409769    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.409985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.409919    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.410059 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.409990    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.410059 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.410041    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwbgh\" (UniqueName: \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.410123 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.410098    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511432 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511345    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511587 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511467    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511587 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511491    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511587 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511523    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511587 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511549    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.511745 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.511586    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbgh\" (UniqueName: \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.512229 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.512197    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.512356 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.512259    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.512649 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.512626    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.515986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.515248    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.523775 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.523739    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.568069 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.568032    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbgh\" (UniqueName: \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\") pod \"premium-w5-96c052-kserve-7985896d76-w4pbv\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") " pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:29.849153 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:29.849056    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:30.224414 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:30.224389    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 21:48:30.226207 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:48:30.226179    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode421aade_93e3_4df1_8f7d_d4201f44944c.slice/crio-d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa WatchSource:0}: Error finding container d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa: Status 404 returned error can't find the container with id d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa
Nov 02 21:48:30.612941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:30.612849    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" event={"ID":"e421aade-93e3-4df1-8f7d-d4201f44944c","Type":"ContainerStarted","Data":"cbbec0bf5cbfc6c87f9c61328a525c1f5f625dd21ee5da50e10f21b1567fda09"}
Nov 02 21:48:30.612941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:30.612890    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" event={"ID":"e421aade-93e3-4df1-8f7d-d4201f44944c","Type":"ContainerStarted","Data":"d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa"}
Nov 02 21:48:30.726210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:30.726154    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podStartSLOduration=1.7261380530000001 podStartE2EDuration="1.726138053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:48:30.725492317 +0000 UTC m=+809.241988092" watchObservedRunningTime="2026-11-02 21:48:30.726138053 +0000 UTC m=+809.242633825"
Nov 02 21:48:30.850263 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:30.850218    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:39.849681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:39.849643    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:39.861792 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:39.861757    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:40.676503 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:40.676472    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:48:40.968109 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:40.968029    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:48:42.933701 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.933670    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 21:48:42.937780 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.937758    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964332    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964379    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964400    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964521    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964689 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964622    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xmhm\" (UniqueName: \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:42.964689 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:42.964648    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.040824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.040780    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-64438f\"/\"openshift-service-ca.crt\""
Nov 02 21:48:43.040824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.040780    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-64438f\"/\"kube-root-ca.crt\""
Nov 02 21:48:43.040824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.040798    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-64438f\"/\"default-dockercfg-ls6zb\""
Nov 02 21:48:43.041127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.040872    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-64438f\"/\"premium-w3-64438f-kserve-self-signed-certs\""
Nov 02 21:48:43.042965 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.042924    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 21:48:43.065674 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065637    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.065877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065726    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.065877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065764    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.065877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065792    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.065877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065832    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.066243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.065928    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xmhm\" (UniqueName: \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.073380 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.066681    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.073380 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.066737    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.073380 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.068837    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.073380 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.069190    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.077353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.077322    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.123527 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.123490    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xmhm\" (UniqueName: \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\") pod \"premium-w3-64438f-kserve-75dd89888-hqsx6\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") " pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.250489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.250386    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:43.630189 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.630159    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 21:48:43.632318 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:48:43.632281    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod454a151c_25fd_485e_8503_ec363d932a36.slice/crio-f089141d8d63d0a876bbf4464abf6036868cc51b4b6e34f5fc487ed180b7cc7d WatchSource:0}: Error finding container f089141d8d63d0a876bbf4464abf6036868cc51b4b6e34f5fc487ed180b7cc7d: Status 404 returned error can't find the container with id f089141d8d63d0a876bbf4464abf6036868cc51b4b6e34f5fc487ed180b7cc7d
Nov 02 21:48:43.689546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:43.689516    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" event={"ID":"454a151c-25fd-485e-8503-ec363d932a36","Type":"ContainerStarted","Data":"f089141d8d63d0a876bbf4464abf6036868cc51b4b6e34f5fc487ed180b7cc7d"}
Nov 02 21:48:44.699695 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:44.699660    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" event={"ID":"454a151c-25fd-485e-8503-ec363d932a36","Type":"ContainerStarted","Data":"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"}
Nov 02 21:48:44.812830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:44.812771    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podStartSLOduration=2.812758798 podStartE2EDuration="2.812758798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:48:44.811455541 +0000 UTC m=+823.327951314" watchObservedRunningTime="2026-11-02 21:48:44.812758798 +0000 UTC m=+823.329254570"
Nov 02 21:48:45.250611 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:45.250565    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:53.250890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:53.250846    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:53.263272 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:53.263245    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:53.759845 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:53.759816    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:48:56.310242 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:48:56.310200    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:48:59.782967 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:59.782928    5580 generic.go:356] "Generic (PLEG): container finished" podID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerID="e9a0ac87e780a28d3c1de640130351ca97d183e35cbb62a6f8a7d86d378cdb78" exitCode=1
Nov 02 21:48:59.783439 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:59.782991    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerDied","Data":"e9a0ac87e780a28d3c1de640130351ca97d183e35cbb62a6f8a7d86d378cdb78"}
Nov 02 21:48:59.783439 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:59.783413    5580 scope.go:122] "RemoveContainer" containerID="e9a0ac87e780a28d3c1de640130351ca97d183e35cbb62a6f8a7d86d378cdb78"
Nov 02 21:48:59.783845 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:48:59.783831    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:49:00.720316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:49:00.720286    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 21:49:01.796761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:49:01.796724    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerStarted","Data":"1a71652067e5238f4e17a4b2f6a397c4f9571d27daf82fb16aa88d3a621fa6e2"}
Nov 02 21:49:01.797273 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:49:01.796943    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:49:12.807070 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:49:12.807035    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 21:49:26.310574 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:49:26.310533    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:49:39.717641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:49:39.717601    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:49:56.310442 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:49:56.310400    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:50:01.722041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:01.722015    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:50:01.723261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:01.723239    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:50:01.732578 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:01.732552    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:50:01.734447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:01.734426    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:50:06.385669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.385629    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:50:06.386103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.385858    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy" containerID="cri-o://4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b" gracePeriod=30
Nov 02 21:50:06.436604 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.436566    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:50:06.436821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.436798    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy" containerID="cri-o://372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d" gracePeriod=30
Nov 02 21:50:06.439581 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.439547    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:50:06.439883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:06.439848    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy" containerID="cri-o://b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997" gracePeriod=30
Nov 02 21:50:11.719802 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.719766    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:50:11.760459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.760427    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:50:11.770029 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.770000    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:50:11.818134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818097    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.818322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818140    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.818322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818179    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.818322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818236    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.818322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818269    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.818322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818304    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.818808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818769    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.818808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818795    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.818996 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818845    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.818996 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818880    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.818996 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818924    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.818992    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819040    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819074    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819065    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819113    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819158    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819189    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819235    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819241    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819307    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819367    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819404    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819465    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819498    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819524    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\") pod \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\" (UID: \"e61b4e2f-b51b-48e6-9d7d-ead80679ce36\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819560    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819588    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\") pod \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\" (UID: \"c3d5ec43-1709-43d7-97b1-57649fa5c2de\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819688    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819716    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\") pod \"e5f628df-175c-42e9-ac99-682bb262cc59\" (UID: \"e5f628df-175c-42e9-ac99-682bb262cc59\") "
Nov 02 21:50:11.819815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.819748    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.820261    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.820339    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.820469    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.820485    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.821441    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.821497    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.821733    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.821913    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822251    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822655    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822660    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822704    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822722    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822738    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822761    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822776    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822776    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822792    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822826    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822848    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822863    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822878    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822891    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.823106 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.822903    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5f628df-175c-42e9-ac99-682bb262cc59-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.824463 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.823326    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "kube-api-access-5pw9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.824463 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.823349    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.824463 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.823451    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.824463 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.823872    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:11.824677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.824503    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.824677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.824619    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:50:11.824865 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.824829    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "kube-api-access-5ghnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.824988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.824863    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "kube-api-access-pljbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.825228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.825197    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:11.825427 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.825407    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token" pod "e61b4e2f-b51b-48e6-9d7d-ead80679ce36" (UID: "e61b4e2f-b51b-48e6-9d7d-ead80679ce36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.825550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.825535    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo" pod "e5f628df-175c-42e9-ac99-682bb262cc59" (UID: "e5f628df-175c-42e9-ac99-682bb262cc59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:50:11.825609 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.825582    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token" pod "c3d5ec43-1709-43d7-97b1-57649fa5c2de" (UID: "c3d5ec43-1709-43d7-97b1-57649fa5c2de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:50:11.924206 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924167    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924206 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924202    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pw9p\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-kube-api-access-5pw9p\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924206 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924217    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5f628df-175c-42e9-ac99-682bb262cc59-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924226    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924235    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924248    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924256    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ghnf\" (UniqueName: \"kubernetes.io/projected/c3d5ec43-1709-43d7-97b1-57649fa5c2de-kube-api-access-5ghnf\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924264    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924278    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924290    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5ec43-1709-43d7-97b1-57649fa5c2de-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924300    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5f628df-175c-42e9-ac99-682bb262cc59-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924308    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pljbk\" (UniqueName: \"kubernetes.io/projected/e5f628df-175c-42e9-ac99-682bb262cc59-kube-api-access-pljbk\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924316    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b4e2f-b51b-48e6-9d7d-ead80679ce36-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:11.924551 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:11.924324    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5ec43-1709-43d7-97b1-57649fa5c2de-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:50:12.197290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.197258    5580 generic.go:356] "Generic (PLEG): container finished" podID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerID="372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d" exitCode=0
Nov 02 21:50:12.197489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.197305    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"
Nov 02 21:50:12.197489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.197341    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" event={"ID":"c3d5ec43-1709-43d7-97b1-57649fa5c2de","Type":"ContainerDied","Data":"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"}
Nov 02 21:50:12.197489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.197398    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks" event={"ID":"c3d5ec43-1709-43d7-97b1-57649fa5c2de","Type":"ContainerDied","Data":"474c880aa79998cf508c1d0e827962da44ec6544f00ed692fe6f8470ec3201b8"}
Nov 02 21:50:12.197489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.197414    5580 scope.go:122] "RemoveContainer" containerID="372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"
Nov 02 21:50:12.199053 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.199026    5580 generic.go:356] "Generic (PLEG): container finished" podID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerID="4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b" exitCode=0
Nov 02 21:50:12.199154 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.199055    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" event={"ID":"e61b4e2f-b51b-48e6-9d7d-ead80679ce36","Type":"ContainerDied","Data":"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"}
Nov 02 21:50:12.199154 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.199095    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"
Nov 02 21:50:12.199154 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.199126    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt" event={"ID":"e61b4e2f-b51b-48e6-9d7d-ead80679ce36","Type":"ContainerDied","Data":"0775864636ab322dda48d4c13bfd3b25c8697706cee805ecb47e263983463b1f"}
Nov 02 21:50:12.201196 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.200637    5580 generic.go:356] "Generic (PLEG): container finished" podID="e5f628df-175c-42e9-ac99-682bb262cc59" containerID="b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997" exitCode=0
Nov 02 21:50:12.201196 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.200690    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"
Nov 02 21:50:12.201196 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.200734    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" event={"ID":"e5f628df-175c-42e9-ac99-682bb262cc59","Type":"ContainerDied","Data":"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"}
Nov 02 21:50:12.201196 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.200847    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw" event={"ID":"e5f628df-175c-42e9-ac99-682bb262cc59","Type":"ContainerDied","Data":"6698eb4a98b75e2281e60aac5f27ff3fd43084195b602d0b92992a845ce6aacc"}
Nov 02 21:50:12.209997 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.209937    5580 scope.go:122] "RemoveContainer" containerID="372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"
Nov 02 21:50:12.210410 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:50:12.210392    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d\": container with ID starting with 372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d not found: ID does not exist" containerID="372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"
Nov 02 21:50:12.210482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.210420    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d"} err="failed to get container status \"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d\": rpc error: code = NotFound desc = could not find container \"372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d\": container with ID starting with 372c665d37199be2e3837bdd8692a5845a86e3024df06d0958e899b0c9c9ee2d not found: ID does not exist"
Nov 02 21:50:12.210482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.210441    5580 scope.go:122] "RemoveContainer" containerID="4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"
Nov 02 21:50:12.223089 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.223048    5580 scope.go:122] "RemoveContainer" containerID="4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"
Nov 02 21:50:12.223564 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:50:12.223534    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b\": container with ID starting with 4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b not found: ID does not exist" containerID="4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"
Nov 02 21:50:12.223650 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.223573    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b"} err="failed to get container status \"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b\": rpc error: code = NotFound desc = could not find container \"4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b\": container with ID starting with 4671adfa678ca931c4071afe25bf673acb246cc439994af9458cd7e73429932b not found: ID does not exist"
Nov 02 21:50:12.223650 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.223600    5580 scope.go:122] "RemoveContainer" containerID="b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"
Nov 02 21:50:12.232930 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.232901    5580 scope.go:122] "RemoveContainer" containerID="b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"
Nov 02 21:50:12.233284 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:50:12.233258    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997\": container with ID starting with b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997 not found: ID does not exist" containerID="b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"
Nov 02 21:50:12.233371 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.233295    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997"} err="failed to get container status \"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997\": rpc error: code = NotFound desc = could not find container \"b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997\": container with ID starting with b8f0a83ad4f95d640bf486917cb71a74e644bdea0002634ee3753d974d67a997 not found: ID does not exist"
Nov 02 21:50:12.360314 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.360279    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:50:12.363761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.363725    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kkqtt"]
Nov 02 21:50:12.473199 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.473107    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:50:12.476144 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.476116    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5qjbw"]
Nov 02 21:50:12.584294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.584259    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:50:12.588297 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:12.588267    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xx4ks"]
Nov 02 21:50:13.594154 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:13.594120    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" path="/var/lib/kubelet/pods/c3d5ec43-1709-43d7-97b1-57649fa5c2de/volumes"
Nov 02 21:50:13.594785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:13.594763    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" path="/var/lib/kubelet/pods/e5f628df-175c-42e9-ac99-682bb262cc59/volumes"
Nov 02 21:50:13.595266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:13.595244    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" path="/var/lib/kubelet/pods/e61b4e2f-b51b-48e6-9d7d-ead80679ce36/volumes"
Nov 02 21:50:20.013791 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:20.013746    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:50:26.311549 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:50:26.311502    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:50:50.554324 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554289    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 21:50:50.554888 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554862    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy"
Nov 02 21:50:50.554888 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554880    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554913    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554918    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554936    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.554945    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.555072    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3d5ec43-1709-43d7-97b1-57649fa5c2de" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.555091    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5f628df-175c-42e9-ac99-682bb262cc59" containerName="istio-proxy"
Nov 02 21:50:50.555145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.555105    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e61b4e2f-b51b-48e6-9d7d-ead80679ce36" containerName="istio-proxy"
Nov 02 21:50:50.558497 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.558477    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.616564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.616532    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 21:50:50.661640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.661602    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ffe386\"/\"openshift-service-ca.crt\""
Nov 02 21:50:50.661640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.661619    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ffe386\"/\"default-dockercfg-rw9zm\""
Nov 02 21:50:50.661640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.661619    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ffe386\"/\"distinct-w1-ffe386-kserve-self-signed-certs\""
Nov 02 21:50:50.661886 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.661668    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ffe386\"/\"kube-root-ca.crt\""
Nov 02 21:50:50.683631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683596    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.683822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683673    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.683822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683769    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkz6k\" (UniqueName: \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.683822 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683794    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.683998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683825    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.683998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.683885    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.784941    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785024    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkz6k\" (UniqueName: \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785047    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785084    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785141    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785434 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785283    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785611 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785585    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785813 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785636    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.785813 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.785664    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.787281 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.787257    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.787669 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.787652    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.840799 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.840725    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkz6k\" (UniqueName: \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\") pod \"distinct-w1-ffe386-kserve-57c6bdb578-vbk9d\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:50.871539 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:50.871508    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:51.252501 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.252474    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 21:50:51.254618 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:50:51.254590    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod823c7335_280d_4641_8cc1_f6deec541e0f.slice/crio-3cf618aeb061ed503d8e3af0ef782754749cc544833f11dda6fec555ead3fd57 WatchSource:0}: Error finding container 3cf618aeb061ed503d8e3af0ef782754749cc544833f11dda6fec555ead3fd57: Status 404 returned error can't find the container with id 3cf618aeb061ed503d8e3af0ef782754749cc544833f11dda6fec555ead3fd57
Nov 02 21:50:51.423076 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.422938    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" event={"ID":"823c7335-280d-4641-8cc1-f6deec541e0f","Type":"ContainerStarted","Data":"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"}
Nov 02 21:50:51.423292 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.423085    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" event={"ID":"823c7335-280d-4641-8cc1-f6deec541e0f","Type":"ContainerStarted","Data":"3cf618aeb061ed503d8e3af0ef782754749cc544833f11dda6fec555ead3fd57"}
Nov 02 21:50:51.537941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.537820    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podStartSLOduration=1.5378033439999998 podStartE2EDuration="1.537803344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:51.536850334 +0000 UTC m=+950.053346107" watchObservedRunningTime="2026-11-02 21:50:51.537803344 +0000 UTC m=+950.054299117"
Nov 02 21:50:51.717102 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.717064    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:50:51.758755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.758711    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:50:51.762810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.762785    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.768931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.768904    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:50:51.773253 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.773227    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:50:51.773416 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.773344    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.777140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.777118    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.798783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.798695    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.798783 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.798750    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.798951 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.798815    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.798951 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.798879    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799023    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799076 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799065    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb7df\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799132    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799158    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799184    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88twt\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799203    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799265    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799348 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799304    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799348 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799319    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799348 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799335    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799366    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799436    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799490    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799505    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799521    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799541    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799658    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799700    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799757    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mf5n\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.799807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799783    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799845    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.799974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799900    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.799974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.799936    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.821201 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.821166    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:50:51.860772 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.860687    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:50:51.862974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.862933    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:50:51.871950 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.871911    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:50:51.901276 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901238    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901276 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901276    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901527 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901404    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901527 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901445    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901527 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901474    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.901681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901543    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.901681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901587    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901641    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb7df\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.901681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901671    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.901876 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901710    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.901876 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901751    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88twt\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.901876 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901780    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.901876 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901818    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901874    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901901    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901932    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901988    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902018    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902023    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902074    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902103    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902130    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902153    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902222    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902269    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902306    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mf5n\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902335    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902334    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.902486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902362    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902633    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.902907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902832    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.903058 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.902950    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.903126 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903102    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903400    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901711    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903601    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903621    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903831    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903929    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.901898    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.904496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.903880    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.906118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.904556    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.906118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.904900    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.906118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.905250    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.907490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.906809    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.907490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.907092    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.907490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.907358    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.909129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.909108    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.958632 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.958598    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.958814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.958697    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb7df\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\") pod \"maas-default-gateway-openshift-default-65c448b547-t46mj\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:51.958814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.958725    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:51.959205 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.959182    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mf5n\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.959408 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.959389    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-hb5hr\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:51.959908 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:51.959892    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88twt\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\") pod \"maas-default-gateway-openshift-default-65c448b547-2f6fx\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:52.085256 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.085153    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:52.090821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.090795    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:52.110127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.110097    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:52.523306 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.523272    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:50:52.526827 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:50:52.526796    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1964092d_351e_4ec5_95f5_2c525da37198.slice/crio-d9ae1500befdd64be7fabb4d7b35a28dbe5e6f597d8b5e5b648b66130f258fc1 WatchSource:0}: Error finding container d9ae1500befdd64be7fabb4d7b35a28dbe5e6f597d8b5e5b648b66130f258fc1: Status 404 returned error can't find the container with id d9ae1500befdd64be7fabb4d7b35a28dbe5e6f597d8b5e5b648b66130f258fc1
Nov 02 21:50:52.529372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.529341    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.529443 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.529408    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.529480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.529441    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.607734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.607686    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:50:52.615125 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.615095    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:50:52.624200 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.624160    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.624379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.624251    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.624379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.624292    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.625209 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:50:52.625114    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd423c1c1_fcd7_44e4_b61b_71855f41c845.slice/crio-2fcc85e32b0fb95e8a72afe0d0889fed68cab68c15c338dda49d44190619a759 WatchSource:0}: Error finding container 2fcc85e32b0fb95e8a72afe0d0889fed68cab68c15c338dda49d44190619a759: Status 404 returned error can't find the container with id 2fcc85e32b0fb95e8a72afe0d0889fed68cab68c15c338dda49d44190619a759
Nov 02 21:50:52.628100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.627832    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.628100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.627922    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:52.628100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:52.628001    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:50:53.469130 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.469079    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" event={"ID":"1964092d-351e-4ec5-95f5-2c525da37198","Type":"ContainerStarted","Data":"4106174c2d0df2b9438fadae407fcbd23cb805a5cd77e59c2748d98cc1e6539c"}
Nov 02 21:50:53.469770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.469736    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" event={"ID":"1964092d-351e-4ec5-95f5-2c525da37198","Type":"ContainerStarted","Data":"d9ae1500befdd64be7fabb4d7b35a28dbe5e6f597d8b5e5b648b66130f258fc1"}
Nov 02 21:50:53.475614 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.475577    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" event={"ID":"facf7658-9637-4756-b835-ecfa99ebb60e","Type":"ContainerStarted","Data":"e22bc08b9553e03a7ca93e4597c518bd0d7325c6f883e3a02ab8cc79e20f5bbe"}
Nov 02 21:50:53.475897 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.475877    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" event={"ID":"facf7658-9637-4756-b835-ecfa99ebb60e","Type":"ContainerStarted","Data":"4d5cf8ca678b5754f8d05ebb8d40b8a90472dad0b7547ec1a9863217df951673"}
Nov 02 21:50:53.483814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.483777    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" event={"ID":"d423c1c1-fcd7-44e4-b61b-71855f41c845","Type":"ContainerStarted","Data":"3786261ce201bea98964b0aa5412d122e10c18bc6573267898005a140d7a20e8"}
Nov 02 21:50:53.488119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.488090    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" event={"ID":"d423c1c1-fcd7-44e4-b61b-71855f41c845","Type":"ContainerStarted","Data":"2fcc85e32b0fb95e8a72afe0d0889fed68cab68c15c338dda49d44190619a759"}
Nov 02 21:50:53.748078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.744587    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podStartSLOduration=2.744562577 podStartE2EDuration="2.744562577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:53.593884977 +0000 UTC m=+952.110380751" watchObservedRunningTime="2026-11-02 21:50:53.744562577 +0000 UTC m=+952.261058351"
Nov 02 21:50:53.881985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.876445    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podStartSLOduration=2.876425237 podStartE2EDuration="2.876425237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:53.745336906 +0000 UTC m=+952.261832679" watchObservedRunningTime="2026-11-02 21:50:53.876425237 +0000 UTC m=+952.392921012"
Nov 02 21:50:53.881985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:53.877407    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podStartSLOduration=2.877386905 podStartE2EDuration="2.877386905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:50:53.874507196 +0000 UTC m=+952.391002969" watchObservedRunningTime="2026-11-02 21:50:53.877386905 +0000 UTC m=+952.393882678"
Nov 02 21:50:54.088462 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.088369    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:50:54.134738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.134710    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:50:54.135078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.135057    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:50:54.135235 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.102091    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:54.135394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.135368    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:54.135764 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.134139    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:54.135920 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.135897    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:54.136109 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.134595    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:54.136228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.136210    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:50:54.759519 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:54.759467    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:50:55.098034 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.087494    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:55.098034 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.087562    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:55.098034 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.092349    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:55.098034 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.092407    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:55.125121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.125084    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:55.125520 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:55.125389    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:50:56.094993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.094400    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:56.094993 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.094470    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:56.095586 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.095208    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:56.095586 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.095256    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:56.119467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.119424    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:56.119824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:56.119771    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:50:56.389046 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:50:56.388260    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:50:57.110986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.086403    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:57.111644 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.111602    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:57.142169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.142113    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:57.142487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.142456    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:57.142666 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.142651    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:57.142790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:57.142770    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:50:58.098984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.089011    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:58.098984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.089076    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:58.098984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.097369    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:58.098984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.097428    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:58.122983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.117370    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:58.122983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:58.117432    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:50:59.087448 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.087406    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:50:59.087740 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.087708    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:50:59.092364 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.092329    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:50:59.092600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.092573    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:50:59.119446 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.119407    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:50:59.119720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:50:59.119692    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:51:00.086204 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.086163    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:51:00.086699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.086235    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:51:00.091685 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.091652    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 02 21:51:00.091803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.091719    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 02 21:51:00.111184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.111142    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 02 21:51:00.111367 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.111217    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 02 21:51:00.874517 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.872630    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:51:00.895667 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:00.895630    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:51:01.746270 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:01.746238    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:51:01.747453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:01.747430    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:51:01.748266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:01.748245    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:51:01.748828 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:01.748804    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:51:02.102308 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.102215    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:51:02.104279 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.103910    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:51:02.106920 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.106892    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:51:02.137115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.137079    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:51:02.137318 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.137142    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:51:02.231297 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.231266    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:51:02.231976 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.231939    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:51:02.232943 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:02.232906    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:51:09.904583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:09.904548    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 21:51:09.909871 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:09.909843    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:09.961456 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:09.961422    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-fb8112-openshift-default-dockercfg-r4kpc\""
Nov 02 21:51:10.013258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.013224    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 21:51:10.028048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028012    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028063    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028165    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028237    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028355 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028320    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028396 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028357    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028481 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028465    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028541 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028503    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp2mv\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.028609 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.028594    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129554    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129601    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2mv\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129628    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129660    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129680    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129704    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.129856 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129759    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.129860    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130003    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130128    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130299 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130283    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130389 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130361    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130401    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.130509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.130431    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.132215 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.132193    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.132298 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.132263    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.187786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.187753    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.187940 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.187881    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp2mv\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.215365 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.215331    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 21:51:10.221244 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.221214    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.224490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.224458    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:10.275312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.275275    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-64438f-openshift-default-dockercfg-tx22g\""
Nov 02 21:51:10.277394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.277363    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 21:51:10.332149 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332114    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332183    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332231    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332259    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332352    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332383    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332498 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332437    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332539 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332494    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpbt\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.332579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.332552    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434139    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434189    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434204    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434252    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434273    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434303    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434336    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpbt\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434384    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434487    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434636    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434869 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434847    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.434916 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434876    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.435003 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.434931    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.435147 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.435127    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.436633 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.436612    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.436782 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.436762    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.493430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.493361    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.493567 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.493512    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpbt\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.538786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.538746    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:10.654714 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.654671    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 21:51:10.656214 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:51:10.656185    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1ace86_270d_4edc_a433_689ce394e8d0.slice/crio-9d5f31c757095e6d141d4919962528b17bd6469bd46d197740af9a7180467d71 WatchSource:0}: Error finding container 9d5f31c757095e6d141d4919962528b17bd6469bd46d197740af9a7180467d71: Status 404 returned error can't find the container with id 9d5f31c757095e6d141d4919962528b17bd6469bd46d197740af9a7180467d71
Nov 02 21:51:10.659012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.658969    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:10.659097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.659049    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:10.659097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.659080    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:10.784712 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.784543    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" event={"ID":"2e1ace86-270d-4edc-a433-689ce394e8d0","Type":"ContainerStarted","Data":"69fa266b24e998b8e284b6d95bc65cf1edaf40e34e2f7be59898804ad9f906fc"}
Nov 02 21:51:10.784712 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.784598    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" event={"ID":"2e1ace86-270d-4edc-a433-689ce394e8d0","Type":"ContainerStarted","Data":"9d5f31c757095e6d141d4919962528b17bd6469bd46d197740af9a7180467d71"}
Nov 02 21:51:10.908074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.907996    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podStartSLOduration=1.9079771129999998 podStartE2EDuration="1.907977113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:10.903985021 +0000 UTC m=+969.420480796" watchObservedRunningTime="2026-11-02 21:51:10.907977113 +0000 UTC m=+969.424472884"
Nov 02 21:51:10.945752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.945719    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 21:51:10.955035 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.954993    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:10.955238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.955221    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:10.955352 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:10.955338    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:11.226986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.226235    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:11.227181 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.227094    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:11.227181 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.227148    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:11.839006 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.838169    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" event={"ID":"c478823c-48ed-4e69-895c-705a33ad50ba","Type":"ContainerStarted","Data":"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"}
Nov 02 21:51:11.839006 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.838249    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" event={"ID":"c478823c-48ed-4e69-895c-705a33ad50ba","Type":"ContainerStarted","Data":"5e56d3587e6996fef0ff31bc98036e47f3facebd19ffa1a6efe04b1586e3079d"}
Nov 02 21:51:11.956070 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:11.956015    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podStartSLOduration=1.955998862 podStartE2EDuration="1.955998862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:11.953472422 +0000 UTC m=+970.469968192" watchObservedRunningTime="2026-11-02 21:51:11.955998862 +0000 UTC m=+970.472494702"
Nov 02 21:51:12.236981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:12.229755    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:12.236981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:12.229821    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:12.539060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:12.538930    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:12.540219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:12.540199    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:12.540308 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:12.540254    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:13.229468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:13.229429    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:13.230170 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:13.230120    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:13.539519 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:13.539423    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:13.539519 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:13.539473    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:14.228989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:14.228416    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:14.228989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:14.228493    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:14.539179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:14.539082    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:14.539179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:14.539147    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:15.226516 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:15.226476    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:15.226829 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:15.226794    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:15.539237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:15.539142    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:15.539656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:15.539528    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:16.227485 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:16.227359    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:16.227485 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:16.227441    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:16.539734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:16.539639    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:16.539734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:16.539711    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:17.225748 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:17.225705    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 02 21:51:17.225942 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:17.225780    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 02 21:51:17.539372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:17.539270    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 02 21:51:17.539372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:17.539340    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 02 21:51:18.230131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.230100    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:18.230590 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.230565    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:18.231471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.231446    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 21:51:18.756901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.756842    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:18.757438 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.757412    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:18.758677 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:18.758657    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 21:51:26.083648 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.083615    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:51:26.299641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.299608    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 21:51:26.303991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.303951    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.313894 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:51:26.313854    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:51:26.325294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325258    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325465 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325322    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325529 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325469    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcwvd\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325577 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325540    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325613 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325595    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325646 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325630    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325745 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325725    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325787 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325768    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.325838 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.325798    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.361539 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.361461    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 21:51:26.407532 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.407498    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-10fdd5-openshift-default-dockercfg-pg4pr\""
Nov 02 21:51:26.427385 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427340    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcwvd\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427423    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427464    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427496    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427565    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427611    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427660    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427728    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.427841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427773    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.428097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.427883    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.428097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.428055    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.428581 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.428550    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.428790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.428767    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.428852 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.428812    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.431014 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.430933    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.431014 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.430996    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.485116 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.485078    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.485259 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.485202    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcwvd\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:26.619829 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:26.619748    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:27.106984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:27.094575    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 21:51:27.145900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:27.145844    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:27.146667 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:27.146165    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:27.146667 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:27.146225    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:28.032571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:28.032534    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" event={"ID":"4143753d-9a95-487b-81fc-040c44d38c63","Type":"ContainerStarted","Data":"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"}
Nov 02 21:51:28.032840 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:28.032819    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" event={"ID":"4143753d-9a95-487b-81fc-040c44d38c63","Type":"ContainerStarted","Data":"64438b1ddb080eef599c6a2fd1924e67a6d9bab7934f7e43092619be7acdbcc2"}
Nov 02 21:51:28.620768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:28.620733    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:28.622485 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:28.622457    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:28.622644 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:28.622518    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:29.620362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:29.620327    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:29.620554 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:29.620388    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:30.620316 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:30.620282    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:30.620813 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:30.620347    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:31.620604 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:31.620563    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:31.621127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:31.620621    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:32.620868 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:32.620831    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:32.621345 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:32.620887    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:33.620992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:33.620948    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 02 21:51:33.621413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:33.621019    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 02 21:51:34.624729 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:34.624700    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:34.625186 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:34.625080    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:34.625834 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:34.625811    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 21:51:34.793803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:34.793722    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podStartSLOduration=8.793701944 podStartE2EDuration="8.793701944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:28.262022823 +0000 UTC m=+986.778518596" watchObservedRunningTime="2026-11-02 21:51:34.793701944 +0000 UTC m=+993.310197713"
Nov 02 21:51:41.707785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.707748    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 21:51:41.711642 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.711618    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.814563 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.814526    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ffe386-openshift-default-dockercfg-8xxrb\""
Nov 02 21:51:41.816884 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.816853    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 21:51:41.857716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857626    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.857716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857692    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mk2\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.857919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857784    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.857919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857829    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.858013 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857922    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.858013 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.857952    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.858091 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.858038    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.858091 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.858060    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.858162 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.858120    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.959874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.959770    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.959883    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.959928    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mk2\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960001    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960041    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960105    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960137    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960163    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960188    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960273    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960366    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960637 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960501    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960637 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.960793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.960775    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.962459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.962431    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:41.962767 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:41.962748    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:42.017248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.017217    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:42.017418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.017351    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mk2\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:42.026318 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.026286    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:42.456790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.456761    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 21:51:42.459470 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:51:42.459421    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb4a535_9ea2_4ddb_a245_d374744a1c4a.slice/crio-c30f77688f6008dc68e16bfc55531b4a4c6b4561373500e9bd9a203034d8b3f1 WatchSource:0}: Error finding container c30f77688f6008dc68e16bfc55531b4a4c6b4561373500e9bd9a203034d8b3f1: Status 404 returned error can't find the container with id c30f77688f6008dc68e16bfc55531b4a4c6b4561373500e9bd9a203034d8b3f1
Nov 02 21:51:42.461480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.461445    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:42.461575 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.461517    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:42.461575 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:42.461545    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:51:43.120295 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:43.120256    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" event={"ID":"bfb4a535-9ea2-4ddb-a245-d374744a1c4a","Type":"ContainerStarted","Data":"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"}
Nov 02 21:51:43.120800 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:43.120338    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" event={"ID":"bfb4a535-9ea2-4ddb-a245-d374744a1c4a","Type":"ContainerStarted","Data":"c30f77688f6008dc68e16bfc55531b4a4c6b4561373500e9bd9a203034d8b3f1"}
Nov 02 21:51:43.235600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:43.235545    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podStartSLOduration=2.235533268 podStartE2EDuration="2.235533268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:51:43.234335456 +0000 UTC m=+1001.750831269" watchObservedRunningTime="2026-11-02 21:51:43.235533268 +0000 UTC m=+1001.752029039"
Nov 02 21:51:44.026714 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:44.026677    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:44.028036 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:44.028009    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:44.028157 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:44.028063    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:45.027139 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:45.027108    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:45.027564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:45.027158    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:46.027696 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:46.027663    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:46.028198 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:46.027722    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:47.028537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:47.028497    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:47.030460 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:47.030420    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:48.027000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:48.026971    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:48.027287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:48.027260    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:49.029863 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:49.029824    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 02 21:51:49.031165 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:49.029881    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 02 21:51:50.031321 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:50.031283    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:50.031725 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:50.031699    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:50.032838 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:50.032815    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 21:51:56.310619 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:51:56.310574    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:51:59.916122 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:51:59.916087    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:52:25.935345 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:25.935308    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:52:26.311259 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:52:26.311175    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:52:39.721731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.721679    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 21:52:39.729827 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.729795    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.780331 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.780290    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 21:52:39.807704 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.807663    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.807704 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.807705    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.807946 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.807733    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sg6v\" (UniqueName: \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.807946 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.807829    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.808107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.807985    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.808160 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.808133    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.826652 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.826612    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"distinct-w0-f48c4d-kserve-self-signed-certs\""
Nov 02 21:52:39.909506 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909464    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909524    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909546    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sg6v\" (UniqueName: \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909585    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909678    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.909783    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910414 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.910120    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910414 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.910400    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.910756 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.910726    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.913049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.913007    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.913429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.913404    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:39.966931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:39.966900    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sg6v\" (UniqueName: \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\") pod \"distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:40.042531 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:40.042431    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:40.469679 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:40.469655    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 21:52:40.472394 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:52:40.472363    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d56c8c7_0d51_4029_84a1_f8fe0ae478e3.slice/crio-c625158dddb4b779fcc296ec778dfd42b25002ea8b3964c63de7e56ffc9417aa WatchSource:0}: Error finding container c625158dddb4b779fcc296ec778dfd42b25002ea8b3964c63de7e56ffc9417aa: Status 404 returned error can't find the container with id c625158dddb4b779fcc296ec778dfd42b25002ea8b3964c63de7e56ffc9417aa
Nov 02 21:52:40.497225 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:40.497195    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" event={"ID":"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3","Type":"ContainerStarted","Data":"c625158dddb4b779fcc296ec778dfd42b25002ea8b3964c63de7e56ffc9417aa"}
Nov 02 21:52:41.504465 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:41.504424    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" event={"ID":"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3","Type":"ContainerStarted","Data":"038ccdfacaead892edc504dd6e4b6f83c954cc192b72ee5a9ad00df00d972060"}
Nov 02 21:52:41.618890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:41.618833    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podStartSLOduration=2.61881693 podStartE2EDuration="2.61881693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:52:41.617212654 +0000 UTC m=+1060.133708427" watchObservedRunningTime="2026-11-02 21:52:41.61881693 +0000 UTC m=+1060.135312757"
Nov 02 21:52:42.043326 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:42.043285    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:45.368110 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.368068    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 21:52:45.372260 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.372235    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.474490 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.474450    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ffe386\"/\"distinct-2-w1-ffe386-kserve-self-signed-certs\""
Nov 02 21:52:45.476855 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.476829    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 21:52:45.489203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489164    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.489339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489219    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.489444 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489422    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.489487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489466    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfdq5\" (UniqueName: \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.489584 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489572    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.489637 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.489601    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590370 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590336    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590578 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590471    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590578 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590495    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfdq5\" (UniqueName: \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590578 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590534    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590766 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590683    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.590925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590902    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.591017 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.590920    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.591067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.591029    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.591195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.591177    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.592798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.592769    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.593475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.593456    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.647284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.647208    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfdq5\" (UniqueName: \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\") pod \"distinct-2-w1-ffe386-kserve-747d659b57-ll4mb\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") " pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:45.685360 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:45.685321    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:46.083823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:46.083792    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 21:52:46.085411 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:52:46.085377    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5604e3f_1005_4c74_be6e_6ec235828e9b.slice/crio-30ff02e20d7473e3775ae7912e00128893e14444f1ddc8ec0124ec05a0c2dc60 WatchSource:0}: Error finding container 30ff02e20d7473e3775ae7912e00128893e14444f1ddc8ec0124ec05a0c2dc60: Status 404 returned error can't find the container with id 30ff02e20d7473e3775ae7912e00128893e14444f1ddc8ec0124ec05a0c2dc60
Nov 02 21:52:46.548936 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:46.548891    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" event={"ID":"d5604e3f-1005-4c74-be6e-6ec235828e9b","Type":"ContainerStarted","Data":"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"}
Nov 02 21:52:46.549448 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:46.548948    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" event={"ID":"d5604e3f-1005-4c74-be6e-6ec235828e9b","Type":"ContainerStarted","Data":"30ff02e20d7473e3775ae7912e00128893e14444f1ddc8ec0124ec05a0c2dc60"}
Nov 02 21:52:46.664222 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:46.664166    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podStartSLOduration=1.6641497250000001 podStartE2EDuration="1.664149725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:52:46.66204427 +0000 UTC m=+1065.178540044" watchObservedRunningTime="2026-11-02 21:52:46.664149725 +0000 UTC m=+1065.180645497"
Nov 02 21:52:46.686335 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:46.686294    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:50.043226 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:50.043182    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:50.055508 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:50.055480    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:50.585507 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:50.585475    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:52:55.685467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:55.685422    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:55.698569 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:55.698533    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:52:56.311350 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:52:56.311309    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:52:56.623493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:52:56.623411    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:53:02.663618 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:02.663581    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:53:26.311559 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:53:26.311515    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:53:26.892531 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.892485    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 21:53:26.896476 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.896456    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935279    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935396    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935494    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935528    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935553    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935582    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t22wm\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935613    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935676    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:26.941425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:26.935815    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.036802 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.036767    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.036802 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.036818    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.036867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.036921    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037048    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037082    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037099    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t22wm\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037120    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037186    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037314    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037545 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037520    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037615 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037580    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037615 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037597    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.037921 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.037903    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.039189 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.039165    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.039522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.039498    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.093572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.093535    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.093743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.093670    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t22wm\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.099937 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.099897    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 21:53:27.210723 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.210683    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:27.639370 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.639347    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 21:53:27.641997 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:53:27.641965    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda574a4f3_284a_49cd_ae37_27313ca6c421.slice/crio-dbcc52809d37b3de493c405900838065b90110aeec7eb7844df9ab27cd0a9ebb WatchSource:0}: Error finding container dbcc52809d37b3de493c405900838065b90110aeec7eb7844df9ab27cd0a9ebb: Status 404 returned error can't find the container with id dbcc52809d37b3de493c405900838065b90110aeec7eb7844df9ab27cd0a9ebb
Nov 02 21:53:27.644752 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.644711    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:27.644861 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.644799    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:27.644861 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.644833    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:27.817404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.817361    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" event={"ID":"a574a4f3-284a-49cd-ae37-27313ca6c421","Type":"ContainerStarted","Data":"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"}
Nov 02 21:53:27.817573 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.817436    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" event={"ID":"a574a4f3-284a-49cd-ae37-27313ca6c421","Type":"ContainerStarted","Data":"dbcc52809d37b3de493c405900838065b90110aeec7eb7844df9ab27cd0a9ebb"}
Nov 02 21:53:27.933837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:27.933770    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podStartSLOduration=1.9337513610000001 podStartE2EDuration="1.933751361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:53:27.932506829 +0000 UTC m=+1106.449002603" watchObservedRunningTime="2026-11-02 21:53:27.933751361 +0000 UTC m=+1106.450247136"
Nov 02 21:53:28.211976 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:28.211861    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:28.213449 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:28.213416    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:28.213609 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:28.213470    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:29.212107 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:29.212058    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:29.212621 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:29.212134    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:30.211221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:30.211186    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:30.211419 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:30.211243    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:31.211184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:31.211144    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:31.211605 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:31.211239    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:32.211914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:32.211872    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:32.212343 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:32.211943    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:32.625601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:32.625518    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:53:33.212807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:33.212771    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:33.213326 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:33.212822    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:34.211347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:34.211298    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 02 21:53:34.211674 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:34.211369    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 02 21:53:35.216016 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:35.215947    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:35.216462 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:35.216398    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:35.217330 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:35.217309    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 21:53:42.325991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.325879    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 21:53:42.330300 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.330281    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.355722 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.355686    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.355904 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.355828    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.355922    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.355988    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.356055    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qbr\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.356181    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.356229    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356351 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.356327    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.356388 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.356362    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.418274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.418237    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 21:53:42.457452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457401    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.457697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457505    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.457697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457566    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.457697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457618    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.457697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457643    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.457697 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457687    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qbr\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457792    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457839    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.457895    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458409 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.458312    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458596 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.458576    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.458782    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.458915 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.458832    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.459140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.459121    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.461798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.461764    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.462192 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.462167    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.515307 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.515271    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.515595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.515575    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qbr\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:42.647028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:42.646935    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:43.077189 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.077113    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 21:53:43.080538 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:53:43.080491    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf2b17e_8cd9_47a3_9d91_40a6f4928634.slice/crio-f77345146b14deed8738fb2147e6cf1c32e32a7c19aedff91c9bc3d416bc3521 WatchSource:0}: Error finding container f77345146b14deed8738fb2147e6cf1c32e32a7c19aedff91c9bc3d416bc3521: Status 404 returned error can't find the container with id f77345146b14deed8738fb2147e6cf1c32e32a7c19aedff91c9bc3d416bc3521
Nov 02 21:53:43.084366 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.084318    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:43.084601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.084578    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:43.084731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.084717    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:53:43.918553 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.918512    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" event={"ID":"9bf2b17e-8cd9-47a3-9d91-40a6f4928634","Type":"ContainerStarted","Data":"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"}
Nov 02 21:53:43.918988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:43.918571    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" event={"ID":"9bf2b17e-8cd9-47a3-9d91-40a6f4928634","Type":"ContainerStarted","Data":"f77345146b14deed8738fb2147e6cf1c32e32a7c19aedff91c9bc3d416bc3521"}
Nov 02 21:53:44.102458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:44.102064    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podStartSLOduration=2.102028376 podStartE2EDuration="2.102028376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:53:44.097579328 +0000 UTC m=+1122.614075118" watchObservedRunningTime="2026-11-02 21:53:44.102028376 +0000 UTC m=+1122.618524149"
Nov 02 21:53:44.647161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:44.647124    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:44.648880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:44.648844    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:44.649029 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:44.648894    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:45.647373 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:45.647334    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:45.647788 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:45.647397    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:46.648133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:46.648094    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:46.648526 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:46.648152    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:47.647475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:47.647442    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:47.647660 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:47.647493    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:48.647336 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:48.647300    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:48.647726 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:48.647356    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:49.647606 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:49.647560    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 02 21:53:49.648063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:49.647641    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 02 21:53:50.651969 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:50.651924    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:50.652391 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:50.652195    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:50.653049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:53:50.653029    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 21:53:56.310326 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:53:56.310286    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:54:12.843297 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.843250    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 21:54:12.847829 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.847801    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.947798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.947763    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 21:54:12.986469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986432    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986663 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986482    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986663 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986611    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986791 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986766    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986840    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986891 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986861    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.986933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.986895    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.987055 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.987043    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:12.987103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:12.987081    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgtj\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088261    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088501 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088314    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgtj\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088501 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088471    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088515    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088549    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088721 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088627    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088721 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088675    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088824 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088773    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.088883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088830    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.089015 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.088952    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.089118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.089015    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.089118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.089084    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.089221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.089157    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.089576 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.089559    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.091439 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.091413    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.091732 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.091711    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.149905 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.149831    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.150084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.150053    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgtj\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.163048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.163014    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:13.595533 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.595508    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 21:54:13.596669 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:54:13.596631    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc18d5c_9ec6_4ddf_9ab7_4e892f4bdfb5.slice/crio-81d8c87e0679a6454b76f800fb5e268794ce8c1ed955a1ba7e93f89b35e138c5 WatchSource:0}: Error finding container 81d8c87e0679a6454b76f800fb5e268794ce8c1ed955a1ba7e93f89b35e138c5: Status 404 returned error can't find the container with id 81d8c87e0679a6454b76f800fb5e268794ce8c1ed955a1ba7e93f89b35e138c5
Nov 02 21:54:13.599360 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.599329    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:54:13.599472 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.599409    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:54:13.599472 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:13.599464    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:54:14.111060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.111012    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" event={"ID":"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5","Type":"ContainerStarted","Data":"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"}
Nov 02 21:54:14.111538 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.111122    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" event={"ID":"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5","Type":"ContainerStarted","Data":"81d8c87e0679a6454b76f800fb5e268794ce8c1ed955a1ba7e93f89b35e138c5"}
Nov 02 21:54:14.163245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.163207    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:14.165087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.165055    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:14.165233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.165120    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:14.269386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:14.269327    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podStartSLOduration=2.269312824 podStartE2EDuration="2.269312824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:54:14.26741735 +0000 UTC m=+1152.783913124" watchObservedRunningTime="2026-11-02 21:54:14.269312824 +0000 UTC m=+1152.785808597"
Nov 02 21:54:15.163818 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:15.163780    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:15.164227 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:15.163848    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:16.163811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:16.163778    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:16.164022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:16.163828    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:17.163627 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:17.163590    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:17.163823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:17.163683    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:18.163680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:18.163641    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:18.164177 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:18.163705    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:19.163941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:19.163903    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:19.164413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:19.164005    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:19.928699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:19.928654    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:54:20.163790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:20.163752    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 02 21:54:20.164027 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:20.163818    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 02 21:54:21.259676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:21.259645    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:21.260124 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:21.259971    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:21.260837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:21.260818    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 21:54:23.949137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:23.949097    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 21:54:23.954275 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:23.954245    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.009050 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.009007    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 21:54:24.055146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.055112    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fb8112\"/\"kube-root-ca.crt\""
Nov 02 21:54:24.055146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.055114    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-fb8112\"/\"openshift-service-ca.crt\""
Nov 02 21:54:24.055368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.055148    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fb8112\"/\"default-dockercfg-h76tv\""
Nov 02 21:54:24.072889 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.072848    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.073128 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.072901    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.073128 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.073065    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.073216 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.073134    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.073216 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.073198    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsfct\" (UniqueName: \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.073314 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.073298    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.114233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.114198    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-fb8112\"/\"distinct-2-w6-fb8112-kserve-self-signed-certs\""
Nov 02 21:54:24.174427 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174384    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174435    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174499    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174641 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174565    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174843 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174645    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsfct\" (UniqueName: \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174843 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174739    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.174971 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.174856    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.175179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.175115    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.178315 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.178279    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.178816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.178791    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.179017 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.178799    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.231433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.231355    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsfct\" (UniqueName: \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\") pod \"distinct-2-w6-fb8112-kserve-67897574d8-pb8qt\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") " pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.268179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.268134    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:24.665785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:24.665756    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 21:54:24.667721 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:54:24.667688    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc56242a_50bb_4a30_9e30_6e4b9b99b04a.slice/crio-5ba48a1405980b4903cfc2b77c7cfa2b55f969d3a42dd299bd32eeeffa5f7642 WatchSource:0}: Error finding container 5ba48a1405980b4903cfc2b77c7cfa2b55f969d3a42dd299bd32eeeffa5f7642: Status 404 returned error can't find the container with id 5ba48a1405980b4903cfc2b77c7cfa2b55f969d3a42dd299bd32eeeffa5f7642
Nov 02 21:54:25.186768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:25.186726    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" event={"ID":"fc56242a-50bb-4a30-9e30-6e4b9b99b04a","Type":"ContainerStarted","Data":"323b9492bc60e0a297e9544fb1b9891fc68ebd7c07be634db0dc9c86afe2a181"}
Nov 02 21:54:25.187179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:25.186781    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" event={"ID":"fc56242a-50bb-4a30-9e30-6e4b9b99b04a","Type":"ContainerStarted","Data":"5ba48a1405980b4903cfc2b77c7cfa2b55f969d3a42dd299bd32eeeffa5f7642"}
Nov 02 21:54:25.269339 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:25.269295    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:25.299788 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:25.299728    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podStartSLOduration=2.299710922 podStartE2EDuration="2.299710922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:54:25.297517519 +0000 UTC m=+1163.814013292" watchObservedRunningTime="2026-11-02 21:54:25.299710922 +0000 UTC m=+1163.816206695"
Nov 02 21:54:26.311091 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:54:26.311053    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:54:34.268600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:34.268563    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:34.281324 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:34.281292    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:35.267480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:35.267446    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:54:40.729682 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.729643    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 21:54:40.735097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.735057    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.791108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.791075    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 21:54:40.803625 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.803585    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.803814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.803733    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4kx\" (UniqueName: \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.804015 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.803992    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.804103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.804085    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.804161 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.804141    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.804337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.804318    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.835925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.835887    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f48c4d\"/\"distinct-2-w0-f48c4d-kserve-self-signed-certs\""
Nov 02 21:54:40.908559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.908520    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.908848 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.908677    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.908848 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.908710    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.909084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.908926    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.909084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.909021    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.909198 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.909113    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4kx\" (UniqueName: \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.911375 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.911271    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.911624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.911480    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.911804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.911781    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.919057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.918138    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.919057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.918433    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:40.965538 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:40.965504    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4kx\" (UniqueName: \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\") pod \"distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") " pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:41.051901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:41.051814    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:41.466193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:41.466163    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 21:54:41.467663 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:54:41.467635    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod048eb4a2_d770_4485_aa56_fd39eedae514.slice/crio-d8c922fd6902db579dc11ab0d31fd71363e62d2f844edc2ef8f1268b214d6f62 WatchSource:0}: Error finding container d8c922fd6902db579dc11ab0d31fd71363e62d2f844edc2ef8f1268b214d6f62: Status 404 returned error can't find the container with id d8c922fd6902db579dc11ab0d31fd71363e62d2f844edc2ef8f1268b214d6f62
Nov 02 21:54:42.307986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:42.307874    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" event={"ID":"048eb4a2-d770-4485-aa56-fd39eedae514","Type":"ContainerStarted","Data":"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"}
Nov 02 21:54:42.307986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:42.307927    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" event={"ID":"048eb4a2-d770-4485-aa56-fd39eedae514","Type":"ContainerStarted","Data":"d8c922fd6902db579dc11ab0d31fd71363e62d2f844edc2ef8f1268b214d6f62"}
Nov 02 21:54:42.422207 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:42.422147    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podStartSLOduration=2.42213036 podStartE2EDuration="2.42213036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:54:42.419649973 +0000 UTC m=+1180.936145745" watchObservedRunningTime="2026-11-02 21:54:42.42213036 +0000 UTC m=+1180.938626132"
Nov 02 21:54:43.052346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:43.052308    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:50.216981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.216196    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.585074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.577526    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.599435 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.599364    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.672421 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.672284    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.754998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.753526    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.808762 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.808675    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.857067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.854055    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:54:50.934991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:50.933346    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": context deadline exceeded"
Nov 02 21:54:51.056682 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:51.054789    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:51.070208 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:51.070177    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:51.423453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:54:51.423421    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:54:56.311301 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:54:56.311255    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:55:01.770499 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:01.770471    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:55:01.771014 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:01.770780    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 21:55:01.780855 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:01.780830    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:55:01.781404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:01.781380    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 21:55:11.767594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.767559    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 21:55:11.771812 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.771793    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839533    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839747 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839588    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839791 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839740    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839833 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839804    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839882 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839850    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvgcm\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.839918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839895    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.840000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.839925    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.840080 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.840067    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.840119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.840088    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.865237 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.865196    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 21:55:11.941593 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941551    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.941811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941642    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.941811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941677    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvgcm\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.941811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941746    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.941811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941794    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942055 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941842    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942055 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.941871    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942055 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942001    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942200 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942061    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942378 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942337    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942472 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942454    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942768    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.942950 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.942929    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.943219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.943195    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.944790 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.944721    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.944899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.944863    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.998360 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.998329    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:11.998520 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:11.998429    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvgcm\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:12.085763 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.085672    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:12.296844 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.296802    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 21:55:12.301326 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.301296    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.347778 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.347683    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.347778 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.347724    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.347877    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.347928    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c7hq\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.348042    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348212 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.348089    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348212 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.348165    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348275 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.348260    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.348331 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.348314    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.357743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.357715    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 21:55:12.449535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449496    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449535 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449533    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449566    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449607    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c7hq\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449656    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449692    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449723    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.449809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449801    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450151 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.449889    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.450190    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.450210    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450313 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.450287    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450367 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.450345    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.450624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.450598    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.452045 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.452019    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.452278 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.452261    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.505691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.505652    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.505850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.505778    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c7hq\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:12.524358 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.524331    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 21:55:12.526288 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:55:12.526254    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fd7f81_3fc6_4fe6_8d1c_77a7e842313c.slice/crio-dff828b41acdf695197cca3be4e28f17db70de849f7bb0d00e067e28dcd34f57 WatchSource:0}: Error finding container dff828b41acdf695197cca3be4e28f17db70de849f7bb0d00e067e28dcd34f57: Status 404 returned error can't find the container with id dff828b41acdf695197cca3be4e28f17db70de849f7bb0d00e067e28dcd34f57
Nov 02 21:55:12.528654 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.528619    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:12.528744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.528690    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:12.528744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.528717    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:12.556864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.556828    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" event={"ID":"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c","Type":"ContainerStarted","Data":"dff828b41acdf695197cca3be4e28f17db70de849f7bb0d00e067e28dcd34f57"}
Nov 02 21:55:12.616680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:12.616646    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:13.035394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.035357    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 21:55:13.041112 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.041084    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.060501 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.060469    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 21:55:13.061396 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061360    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061452    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061559 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061541    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061702 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061604    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061702 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061637    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4xsf\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061706    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061789    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061826    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.061923 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.061895    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.064741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.064706    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:13.064868 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.064816    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:13.064936 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.064878    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:13.099590 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.099550    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 21:55:13.163370 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163337    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163411    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163437    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163488    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163522    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163583    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163672    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163737    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163770    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4xsf\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163798    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163849    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.163992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.163924    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.164186 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.164165    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.164285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.164258    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.166194 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.166168    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.166332 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.166283    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.221077 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.221046    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.221249 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.221227    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4xsf\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\") pod \"e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") " pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.370036 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.369394    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:13.565990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.565932    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" event={"ID":"c66118af-5aaf-4729-b97f-ac8b5e424f31","Type":"ContainerStarted","Data":"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"}
Nov 02 21:55:13.566195 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.566020    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" event={"ID":"c66118af-5aaf-4729-b97f-ac8b5e424f31","Type":"ContainerStarted","Data":"0100d660f0b2e0f915344eea97e857c43848ec5d8d3ceefae3a17f00432f1813"}
Nov 02 21:55:13.567992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.567947    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" event={"ID":"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c","Type":"ContainerStarted","Data":"cfca3b9b3e5b488f39a607e5bb2d082342b63796dd9289ccc01af81c478dd27f"}
Nov 02 21:55:13.617820 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.617783    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:13.819597 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.819533    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podStartSLOduration=2.819515786 podStartE2EDuration="2.819515786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:13.818046582 +0000 UTC m=+1212.334542355" watchObservedRunningTime="2026-11-02 21:55:13.819515786 +0000 UTC m=+1212.336011556"
Nov 02 21:55:13.824400 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.823055    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podStartSLOduration=1.823034839 podStartE2EDuration="1.823034839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:13.7024565 +0000 UTC m=+1212.218952289" watchObservedRunningTime="2026-11-02 21:55:13.823034839 +0000 UTC m=+1212.339530614"
Nov 02 21:55:13.846354 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.846315    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 21:55:13.848003 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:55:13.847974    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d692a6c_6165_4ab6_b343_edd544e293d0.slice/crio-7b8c835a2df90d7bfb6d903679aeeba93a6bd3e3237136877c2ef8d451d16488 WatchSource:0}: Error finding container 7b8c835a2df90d7bfb6d903679aeeba93a6bd3e3237136877c2ef8d451d16488: Status 404 returned error can't find the container with id 7b8c835a2df90d7bfb6d903679aeeba93a6bd3e3237136877c2ef8d451d16488
Nov 02 21:55:13.850896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.850860    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:13.851017 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.850949    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:13.851067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:13.851035    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:14.088353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.087059    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:14.089547 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.089508    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:14.089695 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.089577    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:14.619900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.619858    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:14.620368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.620332    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:14.640054 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.640019    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" event={"ID":"0d692a6c-6165-4ab6-b343-edd544e293d0","Type":"ContainerStarted","Data":"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"}
Nov 02 21:55:14.640312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.640298    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" event={"ID":"0d692a6c-6165-4ab6-b343-edd544e293d0","Type":"ContainerStarted","Data":"7b8c835a2df90d7bfb6d903679aeeba93a6bd3e3237136877c2ef8d451d16488"}
Nov 02 21:55:14.788983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:14.785787    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podStartSLOduration=2.785764814 podStartE2EDuration="2.785764814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:14.774514567 +0000 UTC m=+1213.291010340" watchObservedRunningTime="2026-11-02 21:55:14.785764814 +0000 UTC m=+1213.302260588"
Nov 02 21:55:15.091985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.089446    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:15.091985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.089514    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:15.369972 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.369874    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:15.371319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.371288    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:15.371440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.371345    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:15.624040 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.623438    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:15.624040 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:15.623507    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:16.091981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.090846    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:16.091981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.090986    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:16.370445 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.370359    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:16.370600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.370444    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:16.618976 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.618061    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:16.618976 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:16.618122    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:17.089432 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.089394    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:17.089699 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.089668    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:17.371172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.370658    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:17.371172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.370725    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:17.617983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.617083    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:17.617983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.617149    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:17.968883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:17.968840    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:55:18.111982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.100417    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:18.111982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.100486    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:18.371540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.371397    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:18.371540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.371462    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:18.623041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.622690    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:18.623041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:18.622772    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:19.101986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.098397    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:19.101986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.098463    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:19.375044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.373134    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:19.375044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.373198    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:19.620982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.620373    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:19.620982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.620438    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:19.732982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.732287    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:19.788983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:19.788337    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.094488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.094385    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 02 21:55:20.094767 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.094732    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 02 21:55:20.144142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.144088    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.227979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.218224    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.284983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.282066    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.372041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.371757    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:20.372041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.371836    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:20.417981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.417121    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:55:20.587019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.577195    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.587219 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.587081    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.618979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.618524    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:20.618979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.618610    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:20.618979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.618811    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.672980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.670077    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.745984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.736164    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:55:20.745984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.736752    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:55:20.768985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.763113    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.793980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.793087    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.793980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.793696    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:55:20.851146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.851077    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:20.928675 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:20.928585    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.256048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.237587    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:21.256048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.238240    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.256048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.181928    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.256048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.251914    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.288257 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.265203    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.288257 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.286095    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:55:21.288257 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.286208    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:55:21.288257 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.286558    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.290765 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.289378    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:55:21.450989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.450186    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.451989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.451747    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:55:21.631103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.618277    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.693768 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.693730    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:55:21.716907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.716874    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:21.717231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.717186    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:21.746122 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.746080    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.761421 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.761354    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.766210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.766158    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.768044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.768012    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:55:21.768333 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.768314    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:55:21.768608 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.768592    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:55:21.830303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.804690    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:21.855982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.830600    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:21.863041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.857975    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.949982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.930063    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:55:21.949982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:21.944146    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:55:22.055488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.055443    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/health\": context deadline exceeded"
Nov 02 21:55:22.055873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.055831    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:55:22.058918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.058885    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 02 21:55:22.119981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.116294    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 02 21:55:22.202070 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.163123    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.265750 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.257085    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.283245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.267813    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.310996 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.307502    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:22.310996 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.307568    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.327007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.324171    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:22.327007 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.324249    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.388426 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.286946    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.389609 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.389232    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:22.389609 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.389288    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:22.484435 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.484315    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:55:22.742074 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.737183    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.752544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.751763    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.763122 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.763063    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:22.781983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.769065    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.146057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.861624    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:55:23.146057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:22.941584    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.146057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.128633    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.209988 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:55:23.177474    5580 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.121s"
Nov 02 21:55:23.269183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.269127    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.295148 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.272073    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.554981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.520315    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:55:23.554981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.524215    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 02 21:55:23.554981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.524264    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 02 21:55:23.554981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.545443    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:23.554981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.545508    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.564296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.564252    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:23.564720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.564587    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.696055 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.589456    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.751020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.749185    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:23.821043 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.809052    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:55:23.901919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.901863    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:55:23.915139 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.915101    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:23.915447 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:23.915402    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:24.094925 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.094834    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:24.095287 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095270    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:55:24.095450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095439    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:55:24.095575 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095566    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:55:24.095694 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095685    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:55:24.095825 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095816    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:55:24.095975 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.095948    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:55:24.099521 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.099490    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:55:24.108804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.099714    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:55:24.109123 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.109098    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:24.112241 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.112213    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 21:55:24.114134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.113888    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:55:24.118029 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.118006    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:55:24.128428 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.128398    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:55:24.159654 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.159621    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:24.160349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.160086    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:24.161513 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.161451    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 21:55:24.641438 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.641401    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:24.642785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.642741    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:24.643686 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:24.643664    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 21:55:26.311873 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:55:26.311834    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:55:26.838949 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.838916    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 21:55:26.843098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.843068    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.905141 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.905096    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 21:55:26.915928 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.915890    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.916184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.915944    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.916184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916072    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.916420 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916396    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm7xt\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.917580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916460    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.917580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916499    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.917580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916565    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.917580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916647    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:26.917580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:26.916696    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018455    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm7xt\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018504    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018529    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018575    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018641    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.018776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018683    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018784    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.018812    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019058    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019108 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019102    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019321 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019303    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019384 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019359    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019459    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.019753 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.019734    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.021288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.021267    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.021468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.021440    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.074288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.074259    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.074443 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.074296    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm7xt\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.158728 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.158630    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:27.587304 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:55:27.587272    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1fb38fa_c786_4e2c_b929_b03690d8a717.slice/crio-ae0a8a61d8aac224da5ca878a5edf3d9ba705bf15465981ed1dff4569207536c WatchSource:0}: Error finding container ae0a8a61d8aac224da5ca878a5edf3d9ba705bf15465981ed1dff4569207536c: Status 404 returned error can't find the container with id ae0a8a61d8aac224da5ca878a5edf3d9ba705bf15465981ed1dff4569207536c
Nov 02 21:55:27.590290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.590258    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:27.590381 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.590326    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:27.590381 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.590357    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:27.594479 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:27.594458    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 21:55:28.423843 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.423809    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 21:55:28.429696 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.429657    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.534212 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.534168    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" event={"ID":"d1fb38fa-c786-4e2c-b929-b03690d8a717","Type":"ContainerStarted","Data":"551deb0c4c65134f5e7a298f43af60f480f874f737bf9c689959fd2e1768eef0"}
Nov 02 21:55:28.534405 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.534235    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" event={"ID":"d1fb38fa-c786-4e2c-b929-b03690d8a717","Type":"ContainerStarted","Data":"ae0a8a61d8aac224da5ca878a5edf3d9ba705bf15465981ed1dff4569207536c"}
Nov 02 21:55:28.543544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.543509    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.543722 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.543633    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.543722 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.543689    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.543913 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.543884    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.544017 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.543966    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8gdp\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.544115 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.544086    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.544170 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.544157    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.544300 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.544280    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.544359 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.544320    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.545976 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.545943    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 21:55:28.645466 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645426    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645552    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645601    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8gdp\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645653    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645734    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645835    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.645890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.645930    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646045    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646152    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646261 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646198    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646471 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646326    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646750 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646724    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.646847 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.646730    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.648264 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.648239    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.648665 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.648640    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.715062 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.714569    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.715062 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.714778    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8gdp\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\") pod \"e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") " pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:28.748077 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:28.748038    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:29.159155 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.159055    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:29.160593 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.160561    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:29.160738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.160623    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:29.208320 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.208256    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podStartSLOduration=3.20823405 podStartE2EDuration="3.20823405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:28.711006569 +0000 UTC m=+1227.227502344" watchObservedRunningTime="2026-11-02 21:55:29.20823405 +0000 UTC m=+1227.724729824"
Nov 02 21:55:29.212164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.212132    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 21:55:29.213828 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:55:29.213553    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e73f74_8f30_4f2a_a24c_7625ec0b3247.slice/crio-40cf1416ed1bee79b89660f2f8e1d82cacb30727b3097418244fc0136d0b10f1 WatchSource:0}: Error finding container 40cf1416ed1bee79b89660f2f8e1d82cacb30727b3097418244fc0136d0b10f1: Status 404 returned error can't find the container with id 40cf1416ed1bee79b89660f2f8e1d82cacb30727b3097418244fc0136d0b10f1
Nov 02 21:55:29.217012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.216944    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:29.217100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.217048    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:29.217100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.217079    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:29.543357 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.543314    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" event={"ID":"79e73f74-8f30-4f2a-a24c-7625ec0b3247","Type":"ContainerStarted","Data":"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"}
Nov 02 21:55:29.543542 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.543391    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" event={"ID":"79e73f74-8f30-4f2a-a24c-7625ec0b3247","Type":"ContainerStarted","Data":"40cf1416ed1bee79b89660f2f8e1d82cacb30727b3097418244fc0136d0b10f1"}
Nov 02 21:55:29.696981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.681251    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podStartSLOduration=1.6812312710000001 podStartE2EDuration="1.681231271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:29.680436424 +0000 UTC m=+1228.196932198" watchObservedRunningTime="2026-11-02 21:55:29.681231271 +0000 UTC m=+1228.197727046"
Nov 02 21:55:29.748721 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:29.748676    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:30.159404 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:30.159361    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:30.159587 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:30.159428    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:30.749254 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:30.749222    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:30.749658 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:30.749277    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:31.159730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:31.159626    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:31.159730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:31.159692    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:31.748809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:31.748762    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:31.749027 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:31.748836    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:32.161172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:32.159738    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:32.161172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:32.159802    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:32.748570 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:32.748535    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:32.748755 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:32.748598    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:33.159024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:33.158901    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:33.159024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:33.158989    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:33.748935 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:33.748892    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:33.749450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:33.748972    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:34.159461 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:34.159361    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 02 21:55:34.159461 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:34.159441    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 02 21:55:34.748491 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:34.748448    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:34.748678 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:34.748541    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:35.164356 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:35.164270    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:35.164792 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:35.164728    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:35.165767 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:35.165744    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 21:55:35.749068 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:35.749022    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 02 21:55:35.749268 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:35.749112    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 02 21:55:36.753244 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:36.753214    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:36.753758 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:36.753736    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:36.754628 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:36.754606    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 21:55:45.281634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:45.281579    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:45.394984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:45.393123    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.024140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.024078    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.071136 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.071068    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.107935 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.107888    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:55:46.109048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.109012    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:55:46.120137 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.120083    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.121624 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.121589    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.136721 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.136674    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.138158 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.137809    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.274285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.274160    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.290980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.285073    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.654511 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.654078    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:46.707299 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.707177    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": context deadline exceeded"
Nov 02 21:55:46.722950 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:46.722085    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.094568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.094522    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:55:47.119379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.117139    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.135989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.120170    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:55:47.136311 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.136284    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:55:47.163983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.117187    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:55:47.164563 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.128707    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.173988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.165098    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:55:47.186991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.174311    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:55:47.201982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.141026    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.202600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.141072    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.206981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.128655    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.217982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.211441    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:55:47.247140 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.247096    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:55:47.247434 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.247414    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:55:47.375069 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.288086    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.375069 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.290544    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:55:47.406253 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.381557    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.406253 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.383975    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:55:47.406253 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.403998    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:47.738989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.735100    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:55:47.752986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:47.749088    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:55:48.132376 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.092125    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Nov 02 21:55:48.135114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.132735    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:55:48.135114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.133900    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:55:48.181572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.170280    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.181572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.172216    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.204211 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.203165    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.204211 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.203909    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.228304 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.217045    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.337986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.334051    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.412045 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.408652    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.425033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.416401    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:55:48.425368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.415418    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.504248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.504086    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:55:48.504248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.504167    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:48.736101 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:48.736050    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.228981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.225103    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:55:49.243319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.243021    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.269024 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.267753    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.270568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.270525    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:55:49.271409 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.271372    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.275711 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.275652    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.355421 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.355374    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.378842 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.378784    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:55:49.862625 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.862575    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:49.878793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:49.878741    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:55:50.076020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.059036    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:55:50.250329 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.250271    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:50.255712 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.254390    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:50.269128 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.268083    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:50.275730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.275674    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:55:50.290231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.290180    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:55:50.306418 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.306387    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:55:50.306694 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.306681    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:55:50.306835 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.306825    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:55:50.306934 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.306926    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:55:50.307059 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.307049    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:55:50.307150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.307143    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:55:50.307244 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.307236    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:55:50.314430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.314358    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:55:50.322691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.322656    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:55:50.325457 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.325316    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:55:50.328815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:50.328778    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:55:56.338942 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:55:56.338899    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:55:58.059514 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.059473    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 21:55:58.067127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.067089    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.175215 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.175168    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 21:55:58.236914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.236864    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.236976    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237018    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237062    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpww\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237172    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237218    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237273    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237302    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.237524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.237362    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339055    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339097    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339152    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339241    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339315    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339449 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339349    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339539 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339499    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpww\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339616 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339598    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339693 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339666    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339693 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339677    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339741    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339746    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339782    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.339873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.339858    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.341821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.341797    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.342128 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.342108    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.396772 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.396731    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.396926 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.396785    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpww\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:58.681929 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:58.681890    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:55:59.115920 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.115834    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 21:55:59.155779 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:55:59.155741    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5ca6d9_3fb1_4e4a_85eb_a4da3ba673f5.slice/crio-322672d54a902b03d51dfa047afad623f31166dae2c5ba623c0f4d0958ecb7d5 WatchSource:0}: Error finding container 322672d54a902b03d51dfa047afad623f31166dae2c5ba623c0f4d0958ecb7d5: Status 404 returned error can't find the container with id 322672d54a902b03d51dfa047afad623f31166dae2c5ba623c0f4d0958ecb7d5
Nov 02 21:55:59.158175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.158141    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:59.158290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.158229    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:59.158290 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.158271    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:55:59.251938 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.251901    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" event={"ID":"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5","Type":"ContainerStarted","Data":"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"}
Nov 02 21:55:59.252124 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.251979    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" event={"ID":"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5","Type":"ContainerStarted","Data":"322672d54a902b03d51dfa047afad623f31166dae2c5ba623c0f4d0958ecb7d5"}
Nov 02 21:55:59.369986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.369868    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podStartSLOduration=1.369850556 podStartE2EDuration="1.369850556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:55:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:55:59.367991761 +0000 UTC m=+1257.884487535" watchObservedRunningTime="2026-11-02 21:55:59.369850556 +0000 UTC m=+1257.886346330"
Nov 02 21:55:59.683103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:55:59.683052    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:56:00.704325 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:00.704292    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:00.704731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:00.704343    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:01.703170 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:01.703122    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:01.703397 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:01.703195    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:02.692980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:02.690230    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:02.692980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:02.690292    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:03.697487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:03.697442    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:03.698122 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:03.698086    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:04.684464 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:04.684420    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:04.684808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:04.684775    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:05.683242 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:05.683198    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 02 21:56:05.683667 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:05.683270    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 02 21:56:06.686674 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:06.686641    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:56:06.687125 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:06.686947    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:56:06.687857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:06.687834    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 21:56:12.199877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.199679    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 21:56:12.204672 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.204644    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.261779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.261745    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 21:56:12.287018 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.286971    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287096    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287184    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287281    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287308    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287369    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vkww\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287477 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287463    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287630 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287585    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.287630 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.287609    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.311628 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.311582    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:56:12.316113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.316008    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.421983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.402998    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8c2719c1-openshift-default-dockercfg-zq8l2\""
Nov 02 21:56:12.514660 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.514540    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.514970 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.514901    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515132 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515115    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515260    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.515354 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515343    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.515443 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515434    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515516    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515600 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515591    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.515680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515671    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.515757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515749    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515814    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.515905 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.515896    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518124    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518209    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518248    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518317    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vkww\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518364    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fls88\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518413    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518455    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.518725    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.519001    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.519521    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.522776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.519640    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.530688 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.530640    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:56:12.533292 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.533246    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.550980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.546134    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 21:56:12.555690 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.555661    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.572047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.571858    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.592594 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.592557    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 21:56:12.614985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.606606    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.614985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.608233    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vkww\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\") pod \"e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") " pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.623292 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.623258    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.623472 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.623365    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fls88\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.625066 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.624941    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627243 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.623447    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627391 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627307    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627391 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627355    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627510 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627491    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627566 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627534    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627622 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627607    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627633    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.627730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.627711    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.628111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.628089    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.628672 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.628656    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.629046 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.629029    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.631800 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.631650    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.632825 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.632806    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.681681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.681627    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.690488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.689661    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fls88\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\") pod \"e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") " pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.728674 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.728634    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvf54\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.728914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.728709    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.728914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.728879    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.728949    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.729011    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.729055    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729121 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.729093    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729333 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.729136    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.729333 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.729302    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.758087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.758056    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:12.799421 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.799387    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 21:56:12.803880 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.803851    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.822640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.822615    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:12.830279 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830244    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830306    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.830493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830377    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830409    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvf54\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.830493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830446    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830493 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830488    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kwbd\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830528    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.830803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830581    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830616    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.830803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830676    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.830803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830743    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830801    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830912    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.830949    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.831007    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.831052    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.831131 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.831085    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.831681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.831655    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.832598 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.832575    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.832761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.832657    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.833057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.833027    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.833815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.833794    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.835502 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.835474    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.853643 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.853425    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.933826    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.933900    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934023    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934082    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934160    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934323    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934414    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934460    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.934508    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kwbd\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.935838    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.936140    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.936660    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.938187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.937829    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:12.939246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.939124    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 21:56:12.940361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:12.939886    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.034023 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.022449    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.034023 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.022941    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.034023 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.023635    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvf54\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:13.034023 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.027394    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kwbd\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.061214 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.060688    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:13.068918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.068857    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\") pod \"e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") " pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.123345 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.123299    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:13.215734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.215700    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:56:13.242001 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.240169    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 21:56:13.251841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.251807    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.271918 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.270670    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:13.287284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.283746    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.287284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.283857    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.287284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.283901    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.292694 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.290187    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 21:56:13.348039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.347733    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 21:56:13.353227 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:56:13.353195    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c68e7e0_c1f4_4df9_af1a_04c46b3be70f.slice/crio-9418aad11fe3fab5584d4d54c4299520d271ecac9720dbfd43cb1c52aa9bcb53 WatchSource:0}: Error finding container 9418aad11fe3fab5584d4d54c4299520d271ecac9720dbfd43cb1c52aa9bcb53: Status 404 returned error can't find the container with id 9418aad11fe3fab5584d4d54c4299520d271ecac9720dbfd43cb1c52aa9bcb53
Nov 02 21:56:13.358388 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.358346    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.358525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.358447    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.358525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.358492    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374079    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374188    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374247    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374302    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374348    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374421    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374468    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr2mq\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374514    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.375258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.374528    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477145 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477094    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477294 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477234    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477362 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477306    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477416 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477367    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477474 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477441    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477574 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477558    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477652    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr2mq\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477775 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477759    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.477837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.477810    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.479335 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.478759    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.479560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.479536    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.480130 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.480108    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.480374 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.480349    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.481574 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.481545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.493381 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.493337    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.494127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.494097    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.527705 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.527439    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 21:56:13.532158 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:56:13.532125    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf32480_e907_4c7c_a0db_bbb3cf49e8be.slice/crio-b9c01441790b35038694f8b71c85f555c5bfc5522f17c117f8f819b33f95d666 WatchSource:0}: Error finding container b9c01441790b35038694f8b71c85f555c5bfc5522f17c117f8f819b33f95d666: Status 404 returned error can't find the container with id b9c01441790b35038694f8b71c85f555c5bfc5522f17c117f8f819b33f95d666
Nov 02 21:56:13.535366 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.535087    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.535366 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.535166    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.535879 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.535520    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.537223 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.536602    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerStarted","Data":"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"}
Nov 02 21:56:13.537223 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.536659    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerStarted","Data":"df859280e898dcc9175abbde8354e8bf2d77479d8420a0d0fbc48c375f5cca5b"}
Nov 02 21:56:13.545737 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.544860    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" event={"ID":"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f","Type":"ContainerStarted","Data":"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"}
Nov 02 21:56:13.545737 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.544914    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" event={"ID":"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f","Type":"ContainerStarted","Data":"9418aad11fe3fab5584d4d54c4299520d271ecac9720dbfd43cb1c52aa9bcb53"}
Nov 02 21:56:13.546244 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.546221    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr2mq\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.546975 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.546886    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\") pod \"e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") " pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.620988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.620921    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:13.709029 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.708950    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podStartSLOduration=1.7089322 podStartE2EDuration="1.7089322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:13.654154859 +0000 UTC m=+1272.170650633" watchObservedRunningTime="2026-11-02 21:56:13.7089322 +0000 UTC m=+1272.225427975"
Nov 02 21:56:13.709526 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.709501    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 21:56:13.729770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.728027    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.729770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.728113    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.729770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.728152    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:13.763030 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.762684    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:13.823014 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.822926    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:13.829228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:13.829118    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podStartSLOduration=1.8290968429999999 podStartE2EDuration="1.829096843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:13.818653766 +0000 UTC m=+1272.335149575" watchObservedRunningTime="2026-11-02 21:56:13.829096843 +0000 UTC m=+1272.345592616"
Nov 02 21:56:14.082943 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:56:14.082737    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617b5749_2132_4151_b919_aa96a10621ba.slice/crio-957609e9bc8a69c72bfffe9e2ace80eb89921fbb9d6fc7c4d6f5edfef29ac4a4 WatchSource:0}: Error finding container 957609e9bc8a69c72bfffe9e2ace80eb89921fbb9d6fc7c4d6f5edfef29ac4a4: Status 404 returned error can't find the container with id 957609e9bc8a69c72bfffe9e2ace80eb89921fbb9d6fc7c4d6f5edfef29ac4a4
Nov 02 21:56:14.083284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.083257    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 21:56:14.087990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.087823    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:14.089871 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.087911    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:14.089871 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.088185    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:14.645386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.645341    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" event={"ID":"c07cfac2-2198-4366-a122-0085f921e52d","Type":"ContainerStarted","Data":"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"}
Nov 02 21:56:14.646260 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.646233    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" event={"ID":"c07cfac2-2198-4366-a122-0085f921e52d","Type":"ContainerStarted","Data":"ec79810b55ac2f48ea450ccfb668824437cfc3aff9085fd5e4798070f27a2278"}
Nov 02 21:56:14.652441 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.652403    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" event={"ID":"617b5749-2132-4151-b919-aa96a10621ba","Type":"ContainerStarted","Data":"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"}
Nov 02 21:56:14.652608 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.652466    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" event={"ID":"617b5749-2132-4151-b919-aa96a10621ba","Type":"ContainerStarted","Data":"957609e9bc8a69c72bfffe9e2ace80eb89921fbb9d6fc7c4d6f5edfef29ac4a4"}
Nov 02 21:56:14.660072 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.660033    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" event={"ID":"0cf32480-e907-4c7c-a0db-bbb3cf49e8be","Type":"ContainerStarted","Data":"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"}
Nov 02 21:56:14.660269 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.660107    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" event={"ID":"0cf32480-e907-4c7c-a0db-bbb3cf49e8be","Type":"ContainerStarted","Data":"b9c01441790b35038694f8b71c85f555c5bfc5522f17c117f8f819b33f95d666"}
Nov 02 21:56:14.765279 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.765244    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:14.784924 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.784858    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podStartSLOduration=2.784837723 podStartE2EDuration="2.784837723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:14.778337196 +0000 UTC m=+1273.294832971" watchObservedRunningTime="2026-11-02 21:56:14.784837723 +0000 UTC m=+1273.301333497"
Nov 02 21:56:14.826599 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.826558    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:14.826814 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:14.826623    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:15.037282 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.035868    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podStartSLOduration=2.03584568 podStartE2EDuration="2.03584568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:15.03233152 +0000 UTC m=+1273.548827319" watchObservedRunningTime="2026-11-02 21:56:15.03584568 +0000 UTC m=+1273.552341453"
Nov 02 21:56:15.131985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.126729    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:15.131985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.126870    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:15.131985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.126915    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:15.164540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.162874    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podStartSLOduration=3.162854186 podStartE2EDuration="3.162854186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:15.161233101 +0000 UTC m=+1273.677728875" watchObservedRunningTime="2026-11-02 21:56:15.162854186 +0000 UTC m=+1273.679349960"
Nov 02 21:56:15.274402 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.273829    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:15.274402 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.274012    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:15.274402 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.274057    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:15.663144 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.663102    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:15.671155 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.671104    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:15.694984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.670606    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:15.759984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.718741    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:15.784495 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.784198    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:56:15.865990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.865522    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:15.865990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:15.865586    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:16.135983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.128446    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:16.136319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.136276    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:16.288986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.281130    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:16.288986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.281190    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:16.610245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.610197    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:16.622986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.621385    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:16.622986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.621449    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:16.701598 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.697632    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:16.784270 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.781041    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:16.838563 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.838428    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:16.838563 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.838495    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:16.895117 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:16.890488    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:56:17.206981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.198169    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:17.206981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.198245    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:17.233123 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.232014    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.241995 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.241767    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.261022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.259287    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.261022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.260169    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.276301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.272796    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:17.276301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.272859    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:17.304998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.278380    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.304998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.298059    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.365984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.362145    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.421622 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.421577    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.822986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.815414    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:17.893046 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:17.891218    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.178210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.161117    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:18.178210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.161188    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:56:18.178210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.167093    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.178210 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.167156    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:18.218086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.203053    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.218086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.203118    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:18.218086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.203408    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.218086 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.203444    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.235102    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.235325    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.235386    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.236354    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.244024    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.260980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.244320    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:18.261998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.261943    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.263376 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.263342    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.264599 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.264576    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:56:18.264808 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.264794    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:56:18.317382 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.317337    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.470631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.318020    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.470631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.448054    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:18.470631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.448130    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:56:18.470631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.367188    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.490681 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.479891    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.521982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.408512    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.551576 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.551522    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:18.551863 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.519159    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:56:18.659984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.659100    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:18.661441 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.661412    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.661660 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.661633    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:18.683301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.677947    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:18.818718 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.818605    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:56:18.904982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.904797    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:18.965127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.965090    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:18.972988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:18.969026    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:19.163030 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.162927    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:19.163274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.163245    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:19.365085 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.360598    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:19.467049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.464567    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:19.467049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.373190    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:56:19.491039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.470334    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:19.564985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.532274    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:19.564985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.373321    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:56:19.564985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.425641    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:19.564985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.564357    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:19.564985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.564395    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:19.586123 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.579249    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:19.642985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.617776    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:19.672459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.647370    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:19.677984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.672723    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:19.711142 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:19.678532    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.438583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.034272    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:20.439255 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.439202    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:20.439473 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.063148    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:56:20.439665 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.283898    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:20.439833 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.439812    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:20.485998 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.485908    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:20.486421 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.486390    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:56:20.493119 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.486778    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.493568 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.493526    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.563245 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.563195    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:20.616989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.614146    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.732232 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.732131    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:20.746183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.732556    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:20.760760 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.760714    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.783359 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.761058    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" podUID="cb7ac8cf-c178-48ff-922a-8653c7495ef2" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.20:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.914980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.905277    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:20.976175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:20.961350    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": context deadline exceeded"
Nov 02 21:56:21.129039 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.100094    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.148987 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.131104    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:21.172088 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:56:21.149315    5580 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.557s"
Nov 02 21:56:21.177154 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.177115    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:56:21.177512 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.177495    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:21.191203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.191149    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.191510 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.191465    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:21.192744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.192702    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/health\": context deadline exceeded"
Nov 02 21:56:21.192978 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.192935    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": context deadline exceeded"
Nov 02 21:56:21.282135 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.264217    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.282135 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.264280    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:21.353414 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.329403    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.442118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.442068    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:21.465982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.426043    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" podUID="e267af0e-0894-4514-815d-3bed98989354" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:21.466794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.352509    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:21.554175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.445429    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.463350    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.554785    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.500134    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.500159    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.500185    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:21.570986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.557181    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:21.641979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.614479    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:21.641979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.500082    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:21.641979 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.621312    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:21.704284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.645100    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.704284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.645171    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:21.783983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.783545    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:21.892984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.889574    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:21.925981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.901051    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:21.926303 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:21.913439    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.130806 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.110792    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.169977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.169918    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:22.170577 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.170370    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:22.327489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.327443    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.349983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.333928    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:22.574236 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.338077    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.574932 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.558074    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.575439 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.558132    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:22.575846 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.558163    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.580375 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.558187    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:22.604081 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.604025    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.626907    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.628417    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.628458    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.629036    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.629356    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:22.635011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.629666    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:22.639022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.574186    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:22.811175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.810266    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:22.861211 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.832015    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 02 21:56:22.861977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.861916    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 02 21:56:22.964983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:22.961543    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:23.199322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.199283    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:23.199576 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.199552    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:23.299567 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.290174    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:23.299567 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.290241    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:23.311214 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.311120    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:23.398810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.393092    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:56:23.423189 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.423160    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:56:23.437351 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.437320    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:56:23.569080 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.564864    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:56:23.569080 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.564990    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:56:23.600980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.580268    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:23.715605 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.715565    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 02 21:56:23.715896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.715861    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 02 21:56:23.811897 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.811832    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:23.820222 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.820111    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:23.821827 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.821492    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:23.908971 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.893171    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:56:23.922029 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:23.920519    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:56:24.128041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.125439    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 02 21:56:24.128041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.125499    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 02 21:56:24.279273 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.279201    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 02 21:56:24.279732 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.279683    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 02 21:56:24.306580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.305759    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:56:24.306580 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.306072    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" containerID="cri-o://4106174c2d0df2b9438fadae407fcbd23cb805a5cd77e59c2748d98cc1e6539c" gracePeriod=30
Nov 02 21:56:24.330088 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.330044    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:56:24.330468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.330386    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" containerID="cri-o://3786261ce201bea98964b0aa5412d122e10c18bc6573267898005a140d7a20e8" gracePeriod=30
Nov 02 21:56:24.341854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.341808    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:56:24.342480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.342099    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" containerID="cri-o://e22bc08b9553e03a7ca93e4597c518bd0d7325c6f883e3a02ab8cc79e20f5bbe" gracePeriod=30
Nov 02 21:56:24.359134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.359091    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:56:24.363479 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.359376    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" containerID="cri-o://858ce37746ca757f67ecce6f1fa62de2de8a1aa3cb12f4d5d147186a8182fcd3" gracePeriod=30
Nov 02 21:56:24.848689 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.848652    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:24.858308 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.858198    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:24.871796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:24.870842    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 21:56:25.338564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.338532    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:56:25.470136 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.463028    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:25.470136 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.465337    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:25.491935 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.489365    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 21:56:25.525127 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.525095    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:25.526247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.526224    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:25.527994 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.527930    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 21:56:25.639312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.638184    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:25.639312 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.638683    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:25.639705 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.639686    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 21:56:25.778264 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.778227    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:56:25.783252 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.783222    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.851855 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.851817    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:56:25.880520 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.880486    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-00052a2d-openshift-default-dockercfg-mwxsh\""
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.947754    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.947840    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr46x\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.947884    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.947987    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.948059    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.948141    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.948246    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.948734 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.948341    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:25.953909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:25.953695    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.055557 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055521    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055579    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr46x\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055628    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055746    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055851    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.055946    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056061    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056070    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056169    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056393    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056461    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056478    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.056793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056753    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.057410 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.056985    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.059065 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.059042    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.059269 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.059252    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.112648 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.112611    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.112800 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.112745    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr46x\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\") pod \"e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") " pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.360087 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:56:26.359982    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:56:26.405302 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.405263    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:26.924981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.924703    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:56:26.960779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.960461    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:26.960779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.960572    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:26.960779 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.960629    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:26.992188 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:26.992109    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerStarted","Data":"961d4b906b02d8d291530144fbad825f059b42bc9f60498e2a96b15a66fac22a"}
Nov 02 21:56:27.624000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:27.623114    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-86qzd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:56:27.624000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:27.623175    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:56:27.849246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:27.849210    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 21:56:28.285983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.208218    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 21:56:28.285983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.208408    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.285983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.245361    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerStarted","Data":"d60fe37f674f1a4da03520a0fe0541b9c2f70c3c413dc0c3738f3c739021b405"}
Nov 02 21:56:28.325941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.325897    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.372489 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.337166    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:28.378203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.368333    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.378603    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhfr6\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.378792    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.378850    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.378908    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.378937    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.379003    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.379123    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.386169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.379172    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.421980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.408422    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:56:28.421980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.408485    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:56:28.421980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.408558    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:28.526147 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.516809    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podStartSLOduration=3.51678931 podStartE2EDuration="3.51678931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:28.515701284 +0000 UTC m=+1287.032197096" watchObservedRunningTime="2026-11-02 21:56:28.51678931 +0000 UTC m=+1287.033285087"
Nov 02 21:56:28.550111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.547105    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.564307 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.550413    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:28.567987 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.566753    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.583371 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.583323    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.647398 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.647351    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.649613 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.649570    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.743989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.743927    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.765738    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.809062    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.814669    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.814986    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.815179    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhfr6\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.815592    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.815758    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.815917    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.816053    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.833319 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.816863    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.884980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.884931    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.911084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.886545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.921361 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.886795    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.921611 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.909302    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.922858 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.910600    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.923518 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.923469    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:28.947683 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.947540    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.970796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.970713    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:28.989690 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:28.989655    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:29.044372 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.044330    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhfr6\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\") pod \"e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") " pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:29.228615 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.227251    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:29.394353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.371067    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.394353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.371515    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:29.394353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.376680    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:29.394353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.377375    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:29.480031 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.473362    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.552068    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.552551    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.552913    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.552989    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.578053    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.578516    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.584546    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.604084    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:56:29.628691 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.604545    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:56:29.649706 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.648763    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.650510 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.649995    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:29.705857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.704271    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:56:29.705857 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.704348    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:29.831473 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.824093    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.831473 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.824400    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:56:29.946948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.924377    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:29.946948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:29.926218    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:56:30.392141 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.392099    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:30.690088 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.685459    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:56:30.817337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.807422    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:30.895985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.853050    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:30.895985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.853203    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:56:30.895985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.866087    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:30.895985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.870344    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:30.895985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.872267    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:30.922236 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.921164    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:56:30.925309 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.925264    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:30.987056 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:30.967009    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.033000 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.029867    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.161607 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.161550    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.412505 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.412400    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.424749 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.424500    5580 generic.go:356] "Generic (PLEG): container finished" podID="1964092d-351e-4ec5-95f5-2c525da37198" containerID="4106174c2d0df2b9438fadae407fcbd23cb805a5cd77e59c2748d98cc1e6539c" exitCode=0
Nov 02 21:56:31.424749 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.424654    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" event={"ID":"1964092d-351e-4ec5-95f5-2c525da37198","Type":"ContainerDied","Data":"4106174c2d0df2b9438fadae407fcbd23cb805a5cd77e59c2748d98cc1e6539c"}
Nov 02 21:56:31.427974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.427753    5580 generic.go:356] "Generic (PLEG): container finished" podID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerID="858ce37746ca757f67ecce6f1fa62de2de8a1aa3cb12f4d5d147186a8182fcd3" exitCode=0
Nov 02 21:56:31.427974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.427867    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" event={"ID":"6dcd8d5e-dfe5-41c9-a621-bde78828ce84","Type":"ContainerDied","Data":"858ce37746ca757f67ecce6f1fa62de2de8a1aa3cb12f4d5d147186a8182fcd3"}
Nov 02 21:56:31.477187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.477140    5580 generic.go:356] "Generic (PLEG): container finished" podID="facf7658-9637-4756-b835-ecfa99ebb60e" containerID="e22bc08b9553e03a7ca93e4597c518bd0d7325c6f883e3a02ab8cc79e20f5bbe" exitCode=0
Nov 02 21:56:31.477583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.477557    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" event={"ID":"facf7658-9637-4756-b835-ecfa99ebb60e","Type":"ContainerDied","Data":"e22bc08b9553e03a7ca93e4597c518bd0d7325c6f883e3a02ab8cc79e20f5bbe"}
Nov 02 21:56:31.811981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.784738    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.827986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.811167    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.927306 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.860203    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.927841 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.918653    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.928203 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.918685    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.928564 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.919384    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.928898 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.919414    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:31.929192 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.919433    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/health\": context deadline exceeded"
Nov 02 21:56:31.929330 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.923049    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.929656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.926484    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:31.953990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.949406    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-2f6fx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 02 21:56:31.953990 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:31.949470    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 02 21:56:32.129347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.128443    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:32.139021 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.130214    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:32.390044 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.389882    5580 generic.go:356] "Generic (PLEG): container finished" podID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerID="3786261ce201bea98964b0aa5412d122e10c18bc6573267898005a140d7a20e8" exitCode=0
Nov 02 21:56:32.424985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.421199    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:32.830019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.824225    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:32.848981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:32.844259    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" podUID="e267af0e-0894-4514-815d-3bed98989354" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:33.469313 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.423284    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:56:33.499262    5580 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.911s"
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.499394    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.499426    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" event={"ID":"d423c1c1-fcd7-44e4-b61b-71855f41c845","Type":"ContainerDied","Data":"3786261ce201bea98964b0aa5412d122e10c18bc6573267898005a140d7a20e8"}
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.499596    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.499679    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:56:33.543983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.500127    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.680237    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.680629    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.680900    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.681228    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.681557    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-t46mj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.681608    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.683520    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-hb5hr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:56:33.706982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.683566    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:33.707694 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.707639    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.783077 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.716612    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.783077 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.721089    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.783077 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.740821    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:56:33.863985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.829073    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:33.901155 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:33.901107    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s" podUID="e267af0e-0894-4514-815d-3bed98989354" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:34.204088 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.056693    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:56:34.254901 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.233926    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274139    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274234    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274301    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274531    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274610    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274702    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.274947    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.275602 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.275106    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\") pod \"facf7658-9637-4756-b835-ecfa99ebb60e\" (UID: \"facf7658-9637-4756-b835-ecfa99ebb60e\") "
Nov 02 21:56:34.422280 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.422046    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:56:34.434769 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.433631    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.511272 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.511145    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.541248 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.541203    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:34.554028 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.553972    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.559653 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.559600    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:34.717676 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.609716    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:56:34.770242 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.601329    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:34.771042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.771010    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:56:34.771353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.771339    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-g2h6s"
Nov 02 21:56:34.771495 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.736550    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.830881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.809329    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:56:34.830881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.736644    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.830881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.810029    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.830881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.754918    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:34.830881 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.812732    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.832852 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.832777    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facf7658-9637-4756-b835-ecfa99ebb60e-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.833180 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.833160    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.833296 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.833284    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.833385 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.833375    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.833466 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.833456    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facf7658-9637-4756-b835-ecfa99ebb60e-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.833558 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.833547    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.850162 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.849149    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.863231 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.863181    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:56:34.912906 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.909557    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj" event={"ID":"facf7658-9637-4756-b835-ecfa99ebb60e","Type":"ContainerDied","Data":"4d5cf8ca678b5754f8d05ebb8d40b8a90472dad0b7547ec1a9863217df951673"}
Nov 02 21:56:34.912906 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.909686    5580 scope.go:122] "RemoveContainer" containerID="e22bc08b9553e03a7ca93e4597c518bd0d7325c6f883e3a02ab8cc79e20f5bbe"
Nov 02 21:56:34.913488 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.685822    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.736607    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.917986    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.685791    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.685736    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.686406    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.755064    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.936702    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.936995    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr" event={"ID":"d423c1c1-fcd7-44e4-b61b-71855f41c845","Type":"ContainerDied","Data":"2fcc85e32b0fb95e8a72afe0d0889fed68cab68c15c338dda49d44190619a759"}
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.941970    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.942028    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.942353    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.942463    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.942566    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.942598    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.943282    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.943331    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.943421    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.943620    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\") pod \"d423c1c1-fcd7-44e4-b61b-71855f41c845\" (UID: \"d423c1c1-fcd7-44e4-b61b-71855f41c845\") "
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.945174    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d423c1c1-fcd7-44e4-b61b-71855f41c845-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.945197    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.945217    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facf7658-9637-4756-b835-ecfa99ebb60e-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.946456    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.946635    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.947015    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.949363    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:34.972010 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:34.951969    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.012830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.012795    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.022638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.019395    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:56:35.027720 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.026679    5580 scope.go:122] "RemoveContainer" containerID="3786261ce201bea98964b0aa5412d122e10c18bc6573267898005a140d7a20e8"
Nov 02 21:56:35.066467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.054051    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:56:35.066467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.065194    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:56:35.067533 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.067484    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df" pod "facf7658-9637-4756-b835-ecfa99ebb60e" (UID: "facf7658-9637-4756-b835-ecfa99ebb60e"). InnerVolumeSpecName "kube-api-access-nb7df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.068634 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.068608    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb7df\" (UniqueName: \"kubernetes.io/projected/facf7658-9637-4756-b835-ecfa99ebb60e-kube-api-access-nb7df\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.068798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.068783    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.068883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.068873    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.068981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.068969    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.069073 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.069062    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.069150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.069140    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.069224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.069215    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d423c1c1-fcd7-44e4-b61b-71855f41c845-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.073708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.072229    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:56:35.073708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.072652    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:56:35.075032 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.074198    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:56:35.083041 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.083016    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:56:35.087171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.087137    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n" pod "d423c1c1-fcd7-44e4-b61b-71855f41c845" (UID: "d423c1c1-fcd7-44e4-b61b-71855f41c845"). InnerVolumeSpecName "kube-api-access-2mf5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.087743 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.087712    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:56:35.090311 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.090279    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 21:56:35.097819 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.097791    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:56:35.098120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.098107    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:56:35.098239 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.098230    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:56:35.103864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.102895    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:35.103864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.103049    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:35.103864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.103108    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:35.114553 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.106900    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:56:35.114553 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.108197    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:56:35.114553 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.111907    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:56:35.170049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170016    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.170221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170065    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.170221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170131    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.170221 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170199    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.170321 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170239    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.170321 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170267    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.170399 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170365    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.170473 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170445    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.170636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170501    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.170636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170483    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.170636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170533    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.170870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170644    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.170870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170754    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.170870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170784    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.170870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170785    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.171083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170878    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.171083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171041    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.171173 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.170857    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.171228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171184    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.171344 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171238    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.171344 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171276    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.171344 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171313    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.171500 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171343    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.171500 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171408    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\") pod \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\" (UID: \"6dcd8d5e-dfe5-41c9-a621-bde78828ce84\") "
Nov 02 21:56:35.171607 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171569    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\") pod \"1964092d-351e-4ec5-95f5-2c525da37198\" (UID: \"1964092d-351e-4ec5-95f5-2c525da37198\") "
Nov 02 21:56:35.171607 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171569    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:56:35.171706 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.171637    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172835    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172863    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d423c1c1-fcd7-44e4-b61b-71855f41c845-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172879    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mf5n\" (UniqueName: \"kubernetes.io/projected/d423c1c1-fcd7-44e4-b61b-71855f41c845-kube-api-access-2mf5n\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172894    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172906    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172922    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172937    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172967    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172982    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.172997    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1964092d-351e-4ec5-95f5-2c525da37198-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.173012    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.173197    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.173667    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.173844    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.176941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.176168    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "kube-api-access-dq9f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.178761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.178207    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:56:35.178994 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.178948    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "kube-api-access-88twt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.179146 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.179130    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:56:35.180949 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.180925    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo" pod "1964092d-351e-4ec5-95f5-2c525da37198" (UID: "1964092d-351e-4ec5-95f5-2c525da37198"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:56:35.181241 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.181216    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token" pod "6dcd8d5e-dfe5-41c9-a621-bde78828ce84" (UID: "6dcd8d5e-dfe5-41c9-a621-bde78828ce84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275424    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275472    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275489    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88twt\" (UniqueName: \"kubernetes.io/projected/1964092d-351e-4ec5-95f5-2c525da37198-kube-api-access-88twt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275508    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275523    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1964092d-351e-4ec5-95f5-2c525da37198-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.275537 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275537    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.276060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275554    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.276060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275569    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1964092d-351e-4ec5-95f5-2c525da37198-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.276060 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.275584    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq9f6\" (UniqueName: \"kubernetes.io/projected/6dcd8d5e-dfe5-41c9-a621-bde78828ce84-kube-api-access-dq9f6\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:56:35.635788 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.635742    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:56:35.645317 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.645280    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-t46mj"]
Nov 02 21:56:35.953554 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.952471    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" event={"ID":"7e14dba8-4d31-40b5-b273-67ca8d96ddb9","Type":"ContainerStarted","Data":"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"}
Nov 02 21:56:35.953554 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.952536    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" event={"ID":"7e14dba8-4d31-40b5-b273-67ca8d96ddb9","Type":"ContainerStarted","Data":"c6e6acfa35ca84fd6b4ffedfb2cf23de594b83c46a62a2bcbb68fc17732842cd"}
Nov 02 21:56:35.956741 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.956687    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx" event={"ID":"1964092d-351e-4ec5-95f5-2c525da37198","Type":"ContainerDied","Data":"d9ae1500befdd64be7fabb4d7b35a28dbe5e6f597d8b5e5b648b66130f258fc1"}
Nov 02 21:56:35.957092 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.957029    5580 scope.go:122] "RemoveContainer" containerID="4106174c2d0df2b9438fadae407fcbd23cb805a5cd77e59c2748d98cc1e6539c"
Nov 02 21:56:35.959423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.958487    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"
Nov 02 21:56:35.959931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.959588    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"
Nov 02 21:56:35.959931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.959747    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd" event={"ID":"6dcd8d5e-dfe5-41c9-a621-bde78828ce84","Type":"ContainerDied","Data":"696ff23e6cefbcf4a3e6291434d00035f0be69c90558fb27d7dc076989ce69f7"}
Nov 02 21:56:35.970840 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:35.970814    5580 scope.go:122] "RemoveContainer" containerID="858ce37746ca757f67ecce6f1fa62de2de8a1aa3cb12f4d5d147186a8182fcd3"
Nov 02 21:56:36.228807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:36.228755    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:36.230338 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:36.230305    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:36.230480 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:36.230367    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:37.227896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.227858    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:37.228346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.227913    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:37.373823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.373685    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:56:37.413164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.413130    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-hb5hr"]
Nov 02 21:56:37.503826 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.503699    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podStartSLOduration=10.503681063 podStartE2EDuration="10.503681063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:37.501479506 +0000 UTC m=+1296.017975281" watchObservedRunningTime="2026-11-02 21:56:37.503681063 +0000 UTC m=+1296.020176836"
Nov 02 21:56:37.606933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.606890    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" path="/var/lib/kubelet/pods/d423c1c1-fcd7-44e4-b61b-71855f41c845/volumes"
Nov 02 21:56:37.607940 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.607910    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" path="/var/lib/kubelet/pods/facf7658-9637-4756-b835-ecfa99ebb60e/volumes"
Nov 02 21:56:37.614057 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.614027    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:56:37.620410 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.620380    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-2f6fx"]
Nov 02 21:56:37.727598 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.727472    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:56:37.730769 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:37.730740    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-86qzd"]
Nov 02 21:56:38.237984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:38.228846    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:38.237984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:38.228922    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:39.235981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:39.235043    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:39.235981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:39.235123    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:39.622434 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:39.622188    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1964092d-351e-4ec5-95f5-2c525da37198" path="/var/lib/kubelet/pods/1964092d-351e-4ec5-95f5-2c525da37198/volumes"
Nov 02 21:56:39.622803 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:39.622669    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" path="/var/lib/kubelet/pods/6dcd8d5e-dfe5-41c9-a621-bde78828ce84/volumes"
Nov 02 21:56:40.228238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:40.228193    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:40.228432 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:40.228263    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:41.232285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:41.228505    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:41.232285 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:41.228557    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:42.228266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:42.228223    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 02 21:56:42.228420 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:42.228291    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 02 21:56:43.623227 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.623190    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:43.623774 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.623619    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:43.624469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.624446    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 21:56:43.642209 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642166    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:56:43.642854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642833    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy"
Nov 02 21:56:43.642854 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642852    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642866    5580 cpu_manager.go:414] "containerMap: removing container" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642872    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642882    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642889    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642900    5580 cpu_manager.go:414] "containerMap: removing container" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy"
Nov 02 21:56:43.643047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.642905    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy"
Nov 02 21:56:43.643224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.643065    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1964092d-351e-4ec5-95f5-2c525da37198" containerName="istio-proxy"
Nov 02 21:56:43.643224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.643078    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dcd8d5e-dfe5-41c9-a621-bde78828ce84" containerName="istio-proxy"
Nov 02 21:56:43.643224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.643086    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d423c1c1-fcd7-44e4-b61b-71855f41c845" containerName="istio-proxy"
Nov 02 21:56:43.643224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.643094    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="facf7658-9637-4756-b835-ecfa99ebb60e" containerName="istio-proxy"
Nov 02 21:56:43.650540 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.650356    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.703084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.703042    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:56:43.724169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.724120    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a16a02cb-serving-cert\""
Nov 02 21:56:43.795048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.795015    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.795617 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.795590    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.795834 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.795817    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m4xj\" (UniqueName: \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.796357 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.796325    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.898989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.898862    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.899326 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.899296    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.899453 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.899383    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m4xj\" (UniqueName: \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.899522 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.899479    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.900782 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.900753    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.903067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.903005    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.908579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.908545    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.957058 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.957020    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m4xj\" (UniqueName: \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\") pod \"maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:43.969674 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:43.969635    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:44.357445 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.357411    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:56:44.358565 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:56:44.358539    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d47abdb_3202_409a_9917_7424d7b70645.slice/crio-01ee5275fea8d747a2c66e6fe98a2a02a6f01dbee43b1243dabfe8b58eab812d WatchSource:0}: Error finding container 01ee5275fea8d747a2c66e6fe98a2a02a6f01dbee43b1243dabfe8b58eab812d: Status 404 returned error can't find the container with id 01ee5275fea8d747a2c66e6fe98a2a02a6f01dbee43b1243dabfe8b58eab812d
Nov 02 21:56:44.360282 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.360263    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 21:56:44.410358 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.410323    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 21:56:44.416228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.416199    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.472193 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.472150    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 21:56:44.509910 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.509863    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510149 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.509929    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510149 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510064    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510149 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510112    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510317 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510216    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510400 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510378    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510639 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510416    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510639 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510472    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.510639 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.510505    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fds72\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.612571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612477    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.612571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612540    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.612571 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612570    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.612851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612600    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.612851 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612671    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612879    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.612931    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613015    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613098 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613050    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fds72\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613114    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613165    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613376    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613433 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613386    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.613698 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.613670    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.615322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.615300    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.615475 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.615456    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.669304 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.669272    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.669678 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.669335    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fds72\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:44.732770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:44.732732    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:45.086817 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.086773    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" event={"ID":"5d47abdb-3202-409a-9917-7424d7b70645","Type":"ContainerStarted","Data":"01ee5275fea8d747a2c66e6fe98a2a02a6f01dbee43b1243dabfe8b58eab812d"}
Nov 02 21:56:45.162387 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.162355    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 21:56:45.162913 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:56:45.162879    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d30f30a_d820_4f75_ab68_7a70d08658fa.slice/crio-e885c379f7ceaac893a04aaf8b49bbc7db9f89381b7d46dbcb969945cab5e72f WatchSource:0}: Error finding container e885c379f7ceaac893a04aaf8b49bbc7db9f89381b7d46dbcb969945cab5e72f: Status 404 returned error can't find the container with id e885c379f7ceaac893a04aaf8b49bbc7db9f89381b7d46dbcb969945cab5e72f
Nov 02 21:56:45.165971 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.165922    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:45.166093 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.166020    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:45.166093 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.166053    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:45.937043 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.936998    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 21:56:45.943672 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:45.943637    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.026036 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.023501    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 21:56:46.038230 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038158    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038263    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038322    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038363    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038429 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038399    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038621 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038501    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zrjk\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.038788 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.038768    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.067318 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.067278    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 21:56:46.067560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.067347    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z7n66\""
Nov 02 21:56:46.067560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.067434    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 21:56:46.067560 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.067357    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 21:56:46.110337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.110299    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" event={"ID":"5d47abdb-3202-409a-9917-7424d7b70645","Type":"ContainerStarted","Data":"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"}
Nov 02 21:56:46.110527 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.110404    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:46.112113 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.112087    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" event={"ID":"5d30f30a-d820-4f75-ab68-7a70d08658fa","Type":"ContainerStarted","Data":"3e2938c05fdc1fac8f15d43d72c271067d420af7e76bbacdd1d6cd29ba8b000c"}
Nov 02 21:56:46.112246 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.112133    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" event={"ID":"5d30f30a-d820-4f75-ab68-7a70d08658fa","Type":"ContainerStarted","Data":"e885c379f7ceaac893a04aaf8b49bbc7db9f89381b7d46dbcb969945cab5e72f"}
Nov 02 21:56:46.140726 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.140679    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.140927 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.140788    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.140927 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.140897    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.141070 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.141004    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.141070 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.141064    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.141169 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.141108    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.141297 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.141269    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zrjk\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.142052 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.142020    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.143653 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.143617    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.143796 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.143746    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.144150 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.144119    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.144255 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.144158    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.311061 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.309539    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.340352 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.319497    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zrjk\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rdksh\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.340352 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.321136    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" podStartSLOduration=2.355576267 podStartE2EDuration="3.321116001s" totalImagesPullingTime="965.539734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:46.290409158 +0000 UTC m=+1304.806904931" watchObservedRunningTime="2026-11-02 21:56:46.321116001 +0000 UTC m=+1304.837611774"
Nov 02 21:56:46.420660 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.419476    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podStartSLOduration=2.419454254 podStartE2EDuration="2.419454254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:46.409725369 +0000 UTC m=+1304.926221142" watchObservedRunningTime="2026-11-02 21:56:46.419454254 +0000 UTC m=+1304.935950029"
Nov 02 21:56:46.566657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.562196    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:46.733636 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.733405    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:46.734759 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.734719    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:46.734916 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:46.734783    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:47.121999 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:47.111929    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 21:56:47.155236 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:47.155200    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" event={"ID":"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7","Type":"ContainerStarted","Data":"a04b4d8714fd1cd475f6f2a969b786b0478f9d0aa1eb22f7e9646b56f53988ae"}
Nov 02 21:56:47.733592 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:47.733557    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:47.733754 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:47.733630    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:48.733341 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:48.733303    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:48.733730 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:48.733362    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:49.733714 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:49.733677    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:49.734164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:49.733739    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:50.733603 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:50.733564    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:50.733793 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:50.733625    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:51.733997 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:51.733923    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 02 21:56:51.734390 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:51.734047    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 02 21:56:52.162051 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:52.161413    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:56:52.760393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:52.758645    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:52.760393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:52.760320    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:52.762772 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:52.762715    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 21:56:54.069761 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:54.069727    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:54.070046 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:54.069798    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:56:54.210670 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:54.210633    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" event={"ID":"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7","Type":"ContainerStarted","Data":"2e7b55bbf3c0c7a223e5c104a4307fa85d96a2432d6ed250fa4248ed5ca285db"}
Nov 02 21:56:54.211012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:54.210952    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:54.417840 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:54.417719    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" podStartSLOduration=2.494287614 podStartE2EDuration="9.417699453s" totalImagesPullingTime="6.923411839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:56:54.367607302 +0000 UTC m=+1312.884103098" watchObservedRunningTime="2026-11-02 21:56:54.417699453 +0000 UTC m=+1312.934195225"
Nov 02 21:56:55.232934 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:55.232870    5580 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-rdksh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 21:56:55.233662 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:55.233622    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 21:56:56.312195 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:56:56.312156    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:56:58.221374 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:58.221340    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 21:56:59.549509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.549472    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 21:56:59.553896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.553875    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.639631 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.639590    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 21:56:59.705220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705182    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705406 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705233    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705406 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705366    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705517 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705475    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705517 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705495    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbf7p\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705601 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705570    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705653 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705639    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705760 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705748    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.705798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.705784    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.807656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807552    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.807656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807600    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbf7p\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.807907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807729    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.807907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807798    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807933    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.807996    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808150    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808389 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808364    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808481 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808455    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808719 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808684    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808835 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808752    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.808896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.808850    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.809031 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.809010    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.809299 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.809277    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.810975 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.810920    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.811731 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.811709    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.864847 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.864807    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.864847 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.864822    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbf7p\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:56:59.868442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:56:59.868416    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:57:00.312049 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:00.312020    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 21:57:00.313272 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:57:00.313245    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f98fd7_f614_4f21_be06_d68d441ca799.slice/crio-f15b99428d8647f7d3510bf39c04b0e00c71b2c2322f85833b59256626f18114 WatchSource:0}: Error finding container f15b99428d8647f7d3510bf39c04b0e00c71b2c2322f85833b59256626f18114: Status 404 returned error can't find the container with id f15b99428d8647f7d3510bf39c04b0e00c71b2c2322f85833b59256626f18114
Nov 02 21:57:00.316363 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:00.316325    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:00.316482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:00.316415    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:00.316482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:00.316460    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:01.266206 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.266162    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" event={"ID":"03f98fd7-f614-4f21-be06-d68d441ca799","Type":"ContainerStarted","Data":"45ce881187d21992e88f9f5da8e9c8b81177906e63d50b484e7a1ffcca01da6c"}
Nov 02 21:57:01.266658 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.266241    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" event={"ID":"03f98fd7-f614-4f21-be06-d68d441ca799","Type":"ContainerStarted","Data":"f15b99428d8647f7d3510bf39c04b0e00c71b2c2322f85833b59256626f18114"}
Nov 02 21:57:01.488496 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.488419    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podStartSLOduration=2.488394455 podStartE2EDuration="2.488394455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:57:01.433347927 +0000 UTC m=+1319.949843721" watchObservedRunningTime="2026-11-02 21:57:01.488394455 +0000 UTC m=+1320.004890229"
Nov 02 21:57:01.869117 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.869080    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:57:01.870733 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.870690    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:01.870909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:01.870759    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:02.868870 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:02.868828    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:02.869343 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:02.868905    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:03.869435 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:03.869397    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:03.869907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:03.869471    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:04.869442 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:04.869400    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:04.869657 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:04.869471    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:05.869117 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:05.869078    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:05.869359 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:05.869138    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:06.868893 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:06.868850    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 02 21:57:06.869413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:06.868913    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 02 21:57:08.022926 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:08.022893    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:57:08.023629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:08.023597    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:57:08.024301 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:08.024282    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 21:57:12.833100 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.833048    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:12.861423 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.861374    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:12.862410 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.862076    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:12.876450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.876395    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:12.934293 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.934233    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:57:12.935017 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.934942    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 21:57:12.948266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:12.947398    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:57:13.021151 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.021092    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.073323 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.073220    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.410984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.406578    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:57:13.536983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.531409    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.845980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.840180    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.845980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.840850    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:57:13.863120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.863070    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:57:13.863907 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.863875    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:57:13.884732 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.865783    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.885345 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.881082    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.885924 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.885567    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:57:13.901668 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.901635    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:57:13.975830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.953426    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:13.975830 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.953904    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:57:13.986732 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.978030    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.006048 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.987804    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:57:14.028994 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:13.935068    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:57:14.047096 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.024320    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.064020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.047558    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:57:14.064020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.047636    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:57:14.118680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.074578    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.118680 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.075073    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:57:14.255175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.255109    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.483050 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.414048    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.483050 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.414547    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:57:14.677984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.646594    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:14.749094 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.703397    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:57:15.026355 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:14.905412    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:57:15.051984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.047757    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:57:15.356770 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.356663    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:15.488801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.462164    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:57:15.488801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.463641    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:15.488801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.464095    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:15.538985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.489868    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Nov 02 21:57:15.538985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.503715    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 21:57:15.814982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.811027    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" podUID="cb7ac8cf-c178-48ff-922a-8653c7495ef2" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.20:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:15.908283 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.908136    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:15.969583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.969234    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-4txnq" podUID="bb30c5fd-fa38-4323-ad59-8888963b7224" containerName="main" probeResult="failure" output="Get \"https://10.134.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.057067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.056854    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.057067 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.057034    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-gvpnj" podUID="108043cb-e370-4bb2-961e-191c3414ab62" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.193046 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.160709    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.262610 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.221906    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:57:16.291984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.285847    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Nov 02 21:57:16.320992 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:15.682279    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.338495 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.338456    5580 patch_prober.go:29] interesting pod/thanos-querier-74fc7f6c58-k7tvj container/kube-rbac-proxy-web namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.11:9091/-/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 21:57:16.338794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.338761    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-monitoring/thanos-querier-74fc7f6c58-k7tvj" podUID="3be53397-eac6-47b3-8b63-faa5479e1f3b" containerName="kube-rbac-proxy-web" probeResult="failure" output="Get \"https://10.134.0.11:9091/-/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.364982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.361066    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:57:16.464038 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.463573    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.464213 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.464053    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.488985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.464379    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:57:16.510022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.500755    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.522834 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:16.522777    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:16.649780 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:57:16.649733    5580 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.042s"
Nov 02 21:57:17.161104 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.161055    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.262158    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.263292    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.279681    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.286658    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": context deadline exceeded"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.317725    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.353986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.317882    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": context deadline exceeded"
Nov 02 21:57:17.405984 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.389368    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:57:17.445054 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.406281    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 21:57:17.497981 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.473010    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.523989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.475118    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded"
Nov 02 21:57:17.539991 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.475153    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:57:17.562982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.534228    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.563328 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.557172    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:17.869919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:17.869802    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-dc42x" podUID="cb7ac8cf-c178-48ff-922a-8653c7495ef2" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.20:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.025989 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.025914    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.202118 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.202065    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.264420 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.264365    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.286550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.286507    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 21:57:18.327063 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.327017    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.393130 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.393080    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.426509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.426444    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.507291 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.507185    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 21:57:18.592198 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.592144    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.633980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.540067    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.635398 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.540458    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:18.658122 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:18.625112    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.227052 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.227017    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 21:57:19.266497 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.266455    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.308371 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.306110    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.321111 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.321051    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.375988 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.368283    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 21:57:19.393842 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.393778    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 21:57:19.432019 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.431837    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.642444 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.639479    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 02 21:57:19.644927 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.644887    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.672708 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.672662    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.728315 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.728261    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 21:57:19.767365 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.767322    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 21:57:19.767654 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.767637    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 21:57:19.767801 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.767789    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 21:57:19.767920 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.767909    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 21:57:19.785520 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.785488    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 21:57:19.789980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.787443    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 21:57:19.849327 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.848908    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 21:57:19.850835 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.850807    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 21:57:19.873162 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.872203    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 21:57:19.907874 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:19.907786    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 21:57:26.313489 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:57:26.313426    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:57:27.382864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.382769    5580 generic.go:356] "Generic (PLEG): container finished" podID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerID="e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23" exitCode=0
Nov 02 21:57:27.382864 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.382847    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerDied","Data":"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"}
Nov 02 21:57:27.383412 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.383397    5580 scope.go:122] "RemoveContainer" containerID="e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"
Nov 02 21:57:27.385163 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.385134    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:27.385266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.385219    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:27.385266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:27.385249    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:28.409097 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:28.409056    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerStarted","Data":"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"}
Nov 02 21:57:28.758656 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:28.758618    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:57:28.760143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:28.760115    5580 patch_prober.go:29] interesting pod/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 02 21:57:28.760276 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:28.760183    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 02 21:57:29.759103 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:29.759072    5580 patch_prober.go:29] interesting pod/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 02 21:57:29.759528 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:29.759135    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 02 21:57:30.719266 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:30.719211    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:57:30.759509 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:30.759471    5580 patch_prober.go:29] interesting pod/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 02 21:57:30.759919 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:30.759547    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 02 21:57:31.758495 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:31.758450    5580 patch_prober.go:29] interesting pod/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 02 21:57:31.758686 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:31.758524    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 02 21:57:32.761452 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:32.761416    5580 patch_prober.go:29] interesting pod/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 02 21:57:32.761890 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:32.761497    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 02 21:57:33.809914 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:33.809873    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:57:33.813322 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:33.813295    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:57:44.994419 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:44.994373    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 21:57:45.003809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.003772    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.085912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.085867    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 21:57:45.136337 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136291    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136554 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136431    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136554 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136548    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136679 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136596    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136679 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136643    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136717    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwz9f\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136786 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136758    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136892 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136874    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.136946 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.136933    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.238525 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.238480    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.238764    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.238856    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.238935    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.238983    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwz9f\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239129 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239034    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239132    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239238    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239288    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239395    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239399    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239698 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239464    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239698 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239655    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.239780 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.239736    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.241348 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.241321    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.241750 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.241730    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.295572 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.295503    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.295737 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.295713    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwz9f\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\") pod \"e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") " pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.319776 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.319734    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:45.753021 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.752987    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 21:57:45.795207 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:57:45.795159    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65135e0b_26b5_4bd9_80ff_d30d0f984713.slice/crio-e1c45b0835345be957c0b98686a61b38e8fc214be69e7df3d1c63610229d5aa9 WatchSource:0}: Error finding container e1c45b0835345be957c0b98686a61b38e8fc214be69e7df3d1c63610229d5aa9: Status 404 returned error can't find the container with id e1c45b0835345be957c0b98686a61b38e8fc214be69e7df3d1c63610229d5aa9
Nov 02 21:57:45.798082 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.798032    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:45.798218 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.798123    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:45.798218 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:45.798155    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:57:46.543476 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:46.543422    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" event={"ID":"65135e0b-26b5-4bd9-80ff-d30d0f984713","Type":"ContainerStarted","Data":"2646d78b93c375cfc198f8ef84cd58b573d8147c042baeb46b2591d62a6f6ef0"}
Nov 02 21:57:46.544006 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:46.543590    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" event={"ID":"65135e0b-26b5-4bd9-80ff-d30d0f984713","Type":"ContainerStarted","Data":"e1c45b0835345be957c0b98686a61b38e8fc214be69e7df3d1c63610229d5aa9"}
Nov 02 21:57:46.658986 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:46.658912    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podStartSLOduration=2.658897338 podStartE2EDuration="2.658897338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:57:46.658420576 +0000 UTC m=+1365.174916349" watchObservedRunningTime="2026-11-02 21:57:46.658897338 +0000 UTC m=+1365.175393111"
Nov 02 21:57:47.319941 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:47.319908    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:47.321513 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:47.321485    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:47.321649 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:47.321533    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:48.321014 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:48.320973    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:48.321533 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:48.321046    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:49.322469 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:49.322429    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:49.323134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:49.323102    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:50.321133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:50.321094    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:50.321333 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:50.321152    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:51.320969 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:51.320909    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:51.321487 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:51.321002    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:52.320534 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:52.320498    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 02 21:57:52.320753 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:52.320563    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 02 21:57:53.325425 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:53.325382    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:53.325873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:53.325812    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:53.327233 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:53.327209    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 21:57:56.465450 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:57:56.465415    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:57:58.203346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:57:58.203309    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:58:08.774542 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.774505    5580 generic.go:356] "Generic (PLEG): container finished" podID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerID="d60fe37f674f1a4da03520a0fe0541b9c2f70c3c413dc0c3738f3c739021b405" exitCode=0
Nov 02 21:58:08.775143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.774582    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerDied","Data":"d60fe37f674f1a4da03520a0fe0541b9c2f70c3c413dc0c3738f3c739021b405"}
Nov 02 21:58:08.775238 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.775219    5580 scope.go:122] "RemoveContainer" containerID="d60fe37f674f1a4da03520a0fe0541b9c2f70c3c413dc0c3738f3c739021b405"
Nov 02 21:58:08.777047 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.777018    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:58:08.777132 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.777096    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:58:08.777171 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:08.777138    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:58:09.808200 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:09.808161    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerStarted","Data":"dbed281f8f08b67548813356d1afe30e38372a66f71d08ceaa4bc503dc09d333"}
Nov 02 21:58:10.405794 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:10.405752    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:58:10.406072 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:10.406045    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:58:10.406139 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:10.406101    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:58:11.406382 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:11.406345    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:58:11.406823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:11.406415    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:58:12.406532 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:12.406493    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:58:12.407083 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:12.406575    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:58:13.406031 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:13.405992    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:58:13.406251 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:13.406063    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:58:14.407405 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:14.407357    5580 patch_prober.go:29] interesting pod/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 02 21:58:14.407916 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:14.407436    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 02 21:58:15.410428 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:15.410397    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:58:15.410872 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:15.410695    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:58:15.411513 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:15.411492    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:58:26.313460 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:58:26.313415    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:58:34.408883 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:34.408848    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:58:34.409460 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:34.409108    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy" containerID="cri-o://4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43" gracePeriod=30
Nov 02 21:58:38.263655 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.263615    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:58:38.264244 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.263882    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" podUID="5d47abdb-3202-409a-9917-7424d7b70645" containerName="maas-api" containerID="cri-o://38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6" gracePeriod=30
Nov 02 21:58:38.592985 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.569024    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:58:38.675383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.674458    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\" (UniqueName: \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\") pod \"5d47abdb-3202-409a-9917-7424d7b70645\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") "
Nov 02 21:58:38.675383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.674597    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\") pod \"5d47abdb-3202-409a-9917-7424d7b70645\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") "
Nov 02 21:58:38.675383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.674946    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\") pod \"5d47abdb-3202-409a-9917-7424d7b70645\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") "
Nov 02 21:58:38.676164 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.676141    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\") pod \"5d47abdb-3202-409a-9917-7424d7b70645\" (UID: \"5d47abdb-3202-409a-9917-7424d7b70645\") "
Nov 02 21:58:38.684133 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.676604    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca" pod "5d47abdb-3202-409a-9917-7424d7b70645" (UID: "5d47abdb-3202-409a-9917-7424d7b70645"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:58:38.701104 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.685518    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47abdb-3202-409a-9917-7424d7b70645-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:38.703368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.695123    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs" pod "5d47abdb-3202-409a-9917-7424d7b70645" (UID: "5d47abdb-3202-409a-9917-7424d7b70645"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:58:38.703368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.697795    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls" pod "5d47abdb-3202-409a-9917-7424d7b70645" (UID: "5d47abdb-3202-409a-9917-7424d7b70645"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 21:58:38.703368 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.697896    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj" pod "5d47abdb-3202-409a-9917-7424d7b70645" (UID: "5d47abdb-3202-409a-9917-7424d7b70645"). InnerVolumeSpecName "kube-api-access-5m4xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:58:38.804823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.804720    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:38.804823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.804766    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d47abdb-3202-409a-9917-7424d7b70645-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:38.804823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:38.804777    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m4xj\" (UniqueName: \"kubernetes.io/projected/5d47abdb-3202-409a-9917-7424d7b70645-kube-api-access-5m4xj\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.172466 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.172433    5580 generic.go:356] "Generic (PLEG): container finished" podID="5d47abdb-3202-409a-9917-7424d7b70645" containerID="38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6" exitCode=0
Nov 02 21:58:39.172725 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.172705    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" event={"ID":"5d47abdb-3202-409a-9917-7424d7b70645","Type":"ContainerDied","Data":"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"}
Nov 02 21:58:39.172818 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.172765    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9" event={"ID":"5d47abdb-3202-409a-9917-7424d7b70645","Type":"ContainerDied","Data":"01ee5275fea8d747a2c66e6fe98a2a02a6f01dbee43b1243dabfe8b58eab812d"}
Nov 02 21:58:39.172873 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.172800    5580 scope.go:122] "RemoveContainer" containerID="38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"
Nov 02 21:58:39.173050 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.173033    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"
Nov 02 21:58:39.207274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.207245    5580 scope.go:122] "RemoveContainer" containerID="38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"
Nov 02 21:58:39.207695 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:58:39.207655    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6\": container with ID starting with 38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6 not found: ID does not exist" containerID="38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"
Nov 02 21:58:39.207807 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.207708    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6"} err="failed to get container status \"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6\": rpc error: code = NotFound desc = could not find container \"38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6\": container with ID starting with 38d624407da3bc5b9a8f4b0bdc7ce682ef59c54993fbd8ea7a0b71467287d8a6 not found: ID does not exist"
Nov 02 21:58:39.344340 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.344307    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:58:39.347778 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.347750    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a16a02cb-b7559c9c4-5x8x9"]
Nov 02 21:58:39.608838 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.608332    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d47abdb-3202-409a-9917-7424d7b70645" path="/var/lib/kubelet/pods/5d47abdb-3202-409a-9917-7424d7b70645/volumes"
Nov 02 21:58:39.717395 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.717365    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:58:39.822800 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.822758    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.822832    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.822872    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.822902    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823022 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.822941    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823072    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823247 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823212    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823245    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:58:39.823349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823310    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:58:39.823349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823337    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823503 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823376    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:58:39.823503 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823443    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:58:39.823611 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823513    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\") pod \"36f614a3-cb9a-4d8b-958e-552548f9e6de\" (UID: \"36f614a3-cb9a-4d8b-958e-552548f9e6de\") "
Nov 02 21:58:39.823611 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.823551    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:58:39.824466 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.824439    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.824585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.824470    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36f614a3-cb9a-4d8b-958e-552548f9e6de-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.824585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.824484    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.824585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.824498    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.824585 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.824515    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.825486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.825457    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:58:39.825593 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.825525    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:58:39.825653 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.825628    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "kube-api-access-fls88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:58:39.826078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.826054    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy" pod "36f614a3-cb9a-4d8b-958e-552548f9e6de" (UID: "36f614a3-cb9a-4d8b-958e-552548f9e6de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:58:39.926188 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.926150    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.926188 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.926184    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.926188 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.926193    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fls88\" (UniqueName: \"kubernetes.io/projected/36f614a3-cb9a-4d8b-958e-552548f9e6de-kube-api-access-fls88\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:39.926440 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:39.926204    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36f614a3-cb9a-4d8b-958e-552548f9e6de-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:58:40.181678 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.181644    5580 generic.go:356] "Generic (PLEG): container finished" podID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerID="4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43" exitCode=0
Nov 02 21:58:40.181899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.181727    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"
Nov 02 21:58:40.181899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.181730    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerDied","Data":"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"}
Nov 02 21:58:40.181899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.181785    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc" event={"ID":"36f614a3-cb9a-4d8b-958e-552548f9e6de","Type":"ContainerDied","Data":"df859280e898dcc9175abbde8354e8bf2d77479d8420a0d0fbc48c375f5cca5b"}
Nov 02 21:58:40.181899 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.181807    5580 scope.go:122] "RemoveContainer" containerID="4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"
Nov 02 21:58:40.192809 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.192784    5580 scope.go:122] "RemoveContainer" containerID="e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"
Nov 02 21:58:40.203274 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.203249    5580 scope.go:122] "RemoveContainer" containerID="4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"
Nov 02 21:58:40.203628 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:58:40.203600    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43\": container with ID starting with 4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43 not found: ID does not exist" containerID="4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"
Nov 02 21:58:40.203744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.203641    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43"} err="failed to get container status \"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43\": rpc error: code = NotFound desc = could not find container \"4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43\": container with ID starting with 4a83de507b36c57278e9d0214c50e05a2a0ddeee51ac4ab380e609efe92f1d43 not found: ID does not exist"
Nov 02 21:58:40.203744 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.203668    5580 scope.go:122] "RemoveContainer" containerID="e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"
Nov 02 21:58:40.204034 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:58:40.204012    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23\": container with ID starting with e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23 not found: ID does not exist" containerID="e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"
Nov 02 21:58:40.204084 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.204044    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23"} err="failed to get container status \"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23\": rpc error: code = NotFound desc = could not find container \"e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23\": container with ID starting with e413c1fdb3e1820b82631c0182ad9e6079e10429d931ba6369577e1071557f23 not found: ID does not exist"
Nov 02 21:58:40.349546 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.349512    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:58:40.400749 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:40.400703    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8c2719c1-openshift-default-fdbf6548f-n6ljc"]
Nov 02 21:58:41.607629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:41.607587    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" path="/var/lib/kubelet/pods/36f614a3-cb9a-4d8b-958e-552548f9e6de/volumes"
Nov 02 21:58:44.729815 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.729775    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 21:58:44.730476 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730457    5580 cpu_manager.go:414] "containerMap: removing container" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.730476 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730476    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.730595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730495    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5d47abdb-3202-409a-9917-7424d7b70645" containerName="maas-api"
Nov 02 21:58:44.730595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730500    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d47abdb-3202-409a-9917-7424d7b70645" containerName="maas-api"
Nov 02 21:58:44.730595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730528    5580 cpu_manager.go:414] "containerMap: removing container" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.730595 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730534    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.730804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730632    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.730804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730641    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d47abdb-3202-409a-9917-7424d7b70645" containerName="maas-api"
Nov 02 21:58:44.730804 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.730651    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="36f614a3-cb9a-4d8b-958e-552548f9e6de" containerName="istio-proxy"
Nov 02 21:58:44.734417 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.734386    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:44.791349 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.791308    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 21:58:44.836544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.836503    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-34089f5b\""
Nov 02 21:58:44.836811 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.836691    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-34089f5b-dockercfg-bbjfh\""
Nov 02 21:58:44.856032 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.855002    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:44.856032 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.855242    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnhh8\" (UniqueName: \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:44.918977 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.916610    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 21:58:44.932038 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.922528    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:44.957187 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.957156    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnhh8\" (UniqueName: \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:44.957413 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.957389    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:44.957530 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.957391    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 21:58:44.958410 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:44.958388    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:45.020847 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.019948    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnhh8\" (UniqueName: \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\") pod \"payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") " pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:45.053430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.053368    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:45.059456 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.059419    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mw7j\" (UniqueName: \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.059798 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.059774    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.169042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.167643    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mw7j\" (UniqueName: \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.169042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.167791    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.169302 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.169090    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.228484 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.227212    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mw7j\" (UniqueName: \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\") pod \"payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.259982 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.256591    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:45.469258 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.466644    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 21:58:45.696662 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:45.696618    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 21:58:45.702780 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:58:45.702744    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d4bcbb_fa2a_4735_97e2_71e4b9436dbe.slice/crio-9342a2ab0aab842e57e5374f7d74053605b1a6302ad798e03b506a05080cd9f0 WatchSource:0}: Error finding container 9342a2ab0aab842e57e5374f7d74053605b1a6302ad798e03b506a05080cd9f0: Status 404 returned error can't find the container with id 9342a2ab0aab842e57e5374f7d74053605b1a6302ad798e03b506a05080cd9f0
Nov 02 21:58:46.292132 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:46.292092    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" event={"ID":"170b7e7b-1efd-482b-9739-b486c80b7d32","Type":"ContainerStarted","Data":"1eacba0be03d7c215960336e1c2c0f8925dd0f1b2cf542a509436c198ad68473"}
Nov 02 21:58:46.293346 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:46.293318    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" event={"ID":"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe","Type":"ContainerStarted","Data":"9342a2ab0aab842e57e5374f7d74053605b1a6302ad798e03b506a05080cd9f0"}
Nov 02 21:58:49.324254 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.324209    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" event={"ID":"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe","Type":"ContainerStarted","Data":"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"}
Nov 02 21:58:49.324740 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.324308    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:58:49.326450 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.326427    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" event={"ID":"170b7e7b-1efd-482b-9739-b486c80b7d32","Type":"ContainerStarted","Data":"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"}
Nov 02 21:58:49.326583 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.326558    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:58:49.439033 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.438973    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" podStartSLOduration=2.592443048 podStartE2EDuration="5.438945605s" totalImagesPullingTime="2.846502557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:58:49.435615713 +0000 UTC m=+1427.952111484" watchObservedRunningTime="2026-11-02 21:58:49.438945605 +0000 UTC m=+1427.955441377"
Nov 02 21:58:49.548823 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:58:49.548715    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" podStartSLOduration=2.512581372 podStartE2EDuration="5.548687458s" totalImagesPullingTime="3.036106086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:58:49.546774005 +0000 UTC m=+1428.063269778" watchObservedRunningTime="2026-11-02 21:58:49.548687458 +0000 UTC m=+1428.065183233"
Nov 02 21:58:56.339049 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:58:56.338950    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:59:00.337948 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:00.337920    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 21:59:00.338408 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:00.338193    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 21:59:24.409468 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:24.409428    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:59:24.409974 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:24.409722    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy" containerID="cri-o://dbed281f8f08b67548813356d1afe30e38372a66f71d08ceaa4bc503dc09d333" gracePeriod=30
Nov 02 21:59:26.312930 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:59:26.312890    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:59:29.663816 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.663779    5580 generic.go:356] "Generic (PLEG): container finished" podID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerID="dbed281f8f08b67548813356d1afe30e38372a66f71d08ceaa4bc503dc09d333" exitCode=0
Nov 02 21:59:29.664288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.663856    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerDied","Data":"dbed281f8f08b67548813356d1afe30e38372a66f71d08ceaa4bc503dc09d333"}
Nov 02 21:59:29.664288 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.663941    5580 scope.go:122] "RemoveContainer" containerID="d60fe37f674f1a4da03520a0fe0541b9c2f70c3c413dc0c3738f3c739021b405"
Nov 02 21:59:29.700458 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.700437    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:59:29.777394 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777360    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777441    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777463    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777497    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777579 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777519    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777716    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777810 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777784    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777795    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:29.777922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777818    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.777922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777820    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:29.777922 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.777881    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:29.778179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.778067    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:29.778179 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.778139    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\") pod \"14aac1eb-0401-465c-833d-988ed0ebaddd\" (UID: \"14aac1eb-0401-465c-833d-988ed0ebaddd\") "
Nov 02 21:59:29.778782 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.778722    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 21:59:29.779134 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.779114    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.779228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.779137    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14aac1eb-0401-465c-833d-988ed0ebaddd-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.779228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.779149    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.779228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.779161    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.779228 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.779174    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.780381 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.780341    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 21:59:29.780716 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.780694    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "kube-api-access-hr46x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:29.781109 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.781050    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 21:59:29.781224 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.781152    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy" pod "14aac1eb-0401-465c-833d-988ed0ebaddd" (UID: "14aac1eb-0401-465c-833d-988ed0ebaddd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 21:59:29.882671 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.882632    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.882671 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.882673    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr46x\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-kube-api-access-hr46x\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.882887 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.882688    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:29.882887 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:29.882708    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14aac1eb-0401-465c-833d-988ed0ebaddd-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 21:59:30.672860 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:30.672834    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"
Nov 02 21:59:30.673292 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:30.672877    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs" event={"ID":"14aac1eb-0401-465c-833d-988ed0ebaddd","Type":"ContainerDied","Data":"961d4b906b02d8d291530144fbad825f059b42bc9f60498e2a96b15a66fac22a"}
Nov 02 21:59:30.673292 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:30.672932    5580 scope.go:122] "RemoveContainer" containerID="dbed281f8f08b67548813356d1afe30e38372a66f71d08ceaa4bc503dc09d333"
Nov 02 21:59:30.952446 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:30.952351    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:59:30.955524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:30.955500    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-00052a2d-openshift-default-574cd459b5-wtvrs"]
Nov 02 21:59:31.597511 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.597474    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" path="/var/lib/kubelet/pods/14aac1eb-0401-465c-833d-988ed0ebaddd/volumes"
Nov 02 21:59:31.835240 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.835199    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 21:59:31.836030 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836013    5580 cpu_manager.go:414] "containerMap: removing container" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.836087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836032    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.836087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836047    5580 cpu_manager.go:414] "containerMap: removing container" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.836087 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836051    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.836183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836170    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.836183 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.836181    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="14aac1eb-0401-465c-833d-988ed0ebaddd" containerName="istio-proxy"
Nov 02 21:59:31.841612 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.841593    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:31.894347 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.894309    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 21:59:31.912735 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.912694    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67qv5\" (UniqueName: \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:31.913053 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.913026    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:31.913217 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.913195    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:31.913343 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.913237    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:31.983124 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:31.983091    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a4e90e53-serving-cert\""
Nov 02 21:59:32.014696 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.014659    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67qv5\" (UniqueName: \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.014896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.014727    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.014896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.014749    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.014896 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.014770    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.015103 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:59:32.014894    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-serving-cert: secret "maas-api-e2e-ait-a4e90e53-serving-cert" not found
Nov 02 21:59:32.015103 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:59:32.014996    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls podName:5873d9d3-7d77-4f8a-b024-eab0d8fa62d0 nodeName:}" failed. No retries permitted until 2026-11-02 21:59:32.514971911 +0000 UTC m=+1471.031467678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls") pod "maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" (UID: "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0") : secret "maas-api-e2e-ait-a4e90e53-serving-cert" not found
Nov 02 21:59:32.015536 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.015508    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.017313 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.017292    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.071430 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.071400    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67qv5\" (UniqueName: \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.523945 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.523900    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.526354 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.526331    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") pod \"maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:32.755175 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:32.755138    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:33.138821 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:33.138788    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 21:59:33.140294 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:59:33.140253    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5873d9d3_7d77_4f8a_b024_eab0d8fa62d0.slice/crio-da21fdbc2886baa3377ac438f6d80aac8644bca8a7d44286b42e9087322c7fb4 WatchSource:0}: Error finding container da21fdbc2886baa3377ac438f6d80aac8644bca8a7d44286b42e9087322c7fb4: Status 404 returned error can't find the container with id da21fdbc2886baa3377ac438f6d80aac8644bca8a7d44286b42e9087322c7fb4
Nov 02 21:59:33.695449 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:33.695407    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" event={"ID":"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0","Type":"ContainerStarted","Data":"da21fdbc2886baa3377ac438f6d80aac8644bca8a7d44286b42e9087322c7fb4"}
Nov 02 21:59:34.706184 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:34.706150    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" event={"ID":"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0","Type":"ContainerStarted","Data":"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"}
Nov 02 21:59:34.706654 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:34.706236    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:34.858012 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:34.853914    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" podStartSLOduration=2.889550808 podStartE2EDuration="3.853891702s" totalImagesPullingTime="964.340894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:59:34.832248045 +0000 UTC m=+1473.348743818" watchObservedRunningTime="2026-11-02 21:59:34.853891702 +0000 UTC m=+1473.370387480"
Nov 02 21:59:35.545152 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:35.545113    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 21:59:40.720220 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:40.720187    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 21:59:45.366020 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.365978    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 21:59:45.371640 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.371613    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.426424 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.426365    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 21:59:45.483598 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.480546    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e2be7cc3\""
Nov 02 21:59:45.487034 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.484901    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e2be7cc3-dockercfg-94gqp\""
Nov 02 21:59:45.491729 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.491526    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.491729 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.491573    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz952\" (UniqueName: \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.593202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.593166    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.593202 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.593204    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz952\" (UniqueName: \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.594009 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.593945    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.650120 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.650029    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz952\" (UniqueName: \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:45.690011 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:45.689974    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:46.107897 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:46.107860    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 21:59:46.108691 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:59:46.108651    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a31b1e_ae7e_41f9_b9b0_22ab255a729b.slice/crio-44f445f424436523cee685246d3235076d84f12dafaaaa389e4d77d734656853 WatchSource:0}: Error finding container 44f445f424436523cee685246d3235076d84f12dafaaaa389e4d77d734656853: Status 404 returned error can't find the container with id 44f445f424436523cee685246d3235076d84f12dafaaaa389e4d77d734656853
Nov 02 21:59:46.891969 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:46.891915    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" event={"ID":"82a31b1e-ae7e-41f9-b9b0-22ab255a729b","Type":"ContainerStarted","Data":"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"}
Nov 02 21:59:46.892383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:46.892007    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" event={"ID":"82a31b1e-ae7e-41f9-b9b0-22ab255a729b","Type":"ContainerStarted","Data":"44f445f424436523cee685246d3235076d84f12dafaaaa389e4d77d734656853"}
Nov 02 21:59:46.892383 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:46.892036    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:47.008912 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:47.008850    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" podStartSLOduration=2.008833347 podStartE2EDuration="2.008833347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:59:47.006626279 +0000 UTC m=+1485.523122052" watchObservedRunningTime="2026-11-02 21:59:47.008833347 +0000 UTC m=+1485.525329119"
Nov 02 21:59:56.014255 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.014208    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 21:59:56.018846 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.018814    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.064524 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.064490    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 21:59:56.069235 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.069211    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.076143 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.076107    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qzh4z\""
Nov 02 21:59:56.109099 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109058    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109117    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109255    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109353 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109345    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109528 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109450    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109528 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109515    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2dm7\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109558    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109591    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.109629 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.109621    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.114738 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.114693    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 21:59:56.131931 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.131886    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 21:59:56.211533 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211492    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.211728 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211638    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.211728 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211675    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.211728 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211719    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.211850 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211761    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.211933 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.211912    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212056 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212007    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212056 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212044    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212172 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212066    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212227 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212183    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212236    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2dm7\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212284 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212263    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212284    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212304    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212324    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212393 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212378    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212402    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212416    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qk75\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212524    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212556 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212538    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.212736 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212593    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212736 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212673    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.212736 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.212686    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.214718 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.214685    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.214863 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.214845    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.269909 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.269832    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.270459 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.270437    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2dm7\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\") pod \"maas-default-gateway-openshift-default-65c448b547-5x54n\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.313327 ip-10-0-136-74 kubenswrapper[5580]: E1102 21:59:56.313291    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 21:59:56.314042 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314019    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314095 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314064    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314130 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314095    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314254 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314240    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314305 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314292    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qk75\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314367 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314354    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314409 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314389    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314486 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314459    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314550 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314516    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314612 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314583    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314715 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314691    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314837 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314818    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.314885 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.314844    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.315182 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.315166    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.316544 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.316523    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.316757 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.316741    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.335078 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.335038    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:56.372114 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.372084    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.372278 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.372219    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qk75\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\") pod \"maas-default-gateway-openshift-default-65c448b547-xkdf2\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.386431 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.386395    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:56.768467 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.768438    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 21:59:56.769162 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:59:56.769135    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81913773_6f8a_4d3f_ad54_645788650e2a.slice/crio-84a3f784295ada49a9ab435fb1119988123cdc8cfc9a44fe969452e659af0700 WatchSource:0}: Error finding container 84a3f784295ada49a9ab435fb1119988123cdc8cfc9a44fe969452e659af0700: Status 404 returned error can't find the container with id 84a3f784295ada49a9ab435fb1119988123cdc8cfc9a44fe969452e659af0700
Nov 02 21:59:56.771900 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.771856    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.772006 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.771935    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.772006 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.771984    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.820068 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.820037    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 21:59:56.821375 ip-10-0-136-74 kubenswrapper[5580]: W1102 21:59:56.821345    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a84e52_4615_432d_9bb5_ed52d0203507.slice/crio-88d5938e6269a5ba24bb6e10b48fa0024e0f68310d6c4dd6a84e1601a0b10133 WatchSource:0}: Error finding container 88d5938e6269a5ba24bb6e10b48fa0024e0f68310d6c4dd6a84e1601a0b10133: Status 404 returned error can't find the container with id 88d5938e6269a5ba24bb6e10b48fa0024e0f68310d6c4dd6a84e1601a0b10133
Nov 02 21:59:56.824694 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.824655    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.824785 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.824748    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.824831 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.824790    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 21:59:56.994503 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.994455    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" event={"ID":"98a84e52-4615-432d-9bb5-ed52d0203507","Type":"ContainerStarted","Data":"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"}
Nov 02 21:59:56.994695 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.994530    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" event={"ID":"98a84e52-4615-432d-9bb5-ed52d0203507","Type":"ContainerStarted","Data":"88d5938e6269a5ba24bb6e10b48fa0024e0f68310d6c4dd6a84e1601a0b10133"}
Nov 02 21:59:56.996379 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.996352    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" event={"ID":"81913773-6f8a-4d3f-ad54-645788650e2a","Type":"ContainerStarted","Data":"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"}
Nov 02 21:59:56.996510 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:56.996401    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" event={"ID":"81913773-6f8a-4d3f-ad54-645788650e2a","Type":"ContainerStarted","Data":"84a3f784295ada49a9ab435fb1119988123cdc8cfc9a44fe969452e659af0700"}
Nov 02 21:59:57.113166 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.113060    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podStartSLOduration=2.113042097 podStartE2EDuration="2.113042097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:59:57.107679748 +0000 UTC m=+1495.624175549" watchObservedRunningTime="2026-11-02 21:59:57.113042097 +0000 UTC m=+1495.629537870"
Nov 02 21:59:57.228386 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.228313    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podStartSLOduration=2.228290008 podStartE2EDuration="2.228290008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 21:59:57.222147406 +0000 UTC m=+1495.738643214" watchObservedRunningTime="2026-11-02 21:59:57.228290008 +0000 UTC m=+1495.744785782"
Nov 02 21:59:57.335877 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.335840    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 21:59:57.337482 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.337451    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 21:59:57.337644 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.337504    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 21:59:57.388638 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.388533    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 21:59:57.402980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.391018    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 21:59:57.402980 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.391086    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 21:59:57.932983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:57.928183    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 21:59:58.336727 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:58.336430    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 21:59:58.336727 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:58.336499    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 21:59:58.399983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:58.397006    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 21:59:58.399983 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:58.397099    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 21:59:59.336500 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:59.336459    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 21:59:59.336714 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:59.336526    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 21:59:59.393853 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:59.393816    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 21:59:59.394481 ip-10-0-136-74 kubenswrapper[5580]: I1102 21:59:59.394450    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:00.339269 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:00.338701    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:00:00.339269 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:00.338774    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:00:00.435451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:00.435413    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:00:00.436077 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:00.436046    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:01.335620 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.335583    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:00:01.335810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.335654    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:00:01.387398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.387355    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:00:01.388438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.388405    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:01.958931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.958898    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:00:01.960567 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.960547    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:00:01.970794 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.970774    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:00:01.972222 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:01.972204    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:00:02.337155 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:02.337070    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:00:02.337155 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:02.337127    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:00:02.397982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:02.397023    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:00:02.397982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:02.397097    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:03.336631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.336595    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:00:03.337339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.337301    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:00:03.387695 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.387659    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:00:03.387907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.387717    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:03.748080 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.748031    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:00:03.762316 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.757088    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:03.806584 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.806543    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:00:03.845612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.845576    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-caac72\""
Nov 02 22:00:03.846514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.846487    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-caac72-dockercfg-4vd6p\""
Nov 02 22:00:03.864666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.864278    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:03.864666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.864358    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n58s\" (UniqueName: \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:03.966240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.966190    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:03.966452 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.966279    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n58s\" (UniqueName: \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:03.971416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:03.971376    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:04.024882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.024781    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n58s\" (UniqueName: \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\") pod \"payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:04.085019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.083728    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:04.335763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.335668    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:00:04.335763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.335730    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:00:04.388720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.388596    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:00:04.388720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.388660    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 02 22:00:04.525520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:04.525486    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:00:04.527878 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:00:04.527848    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f9ec66_73df_47c7_9a17_8778899610b7.slice/crio-4167d78beea53cdfe2b8815f2f7a929af7fd55ea70b33295453c5ed045cd64ca WatchSource:0}: Error finding container 4167d78beea53cdfe2b8815f2f7a929af7fd55ea70b33295453c5ed045cd64ca: Status 404 returned error can't find the container with id 4167d78beea53cdfe2b8815f2f7a929af7fd55ea70b33295453c5ed045cd64ca
Nov 02 22:00:05.341731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.341681    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" event={"ID":"68f9ec66-73df-47c7-9a17-8778899610b7","Type":"ContainerStarted","Data":"e08c8d0bad4f3687ca9c3b8ccd285a7bb55f4445f02c69519cdc001369b642f9"}
Nov 02 22:00:05.341919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.341808    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:05.341919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.341840    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" event={"ID":"68f9ec66-73df-47c7-9a17-8778899610b7","Type":"ContainerStarted","Data":"4167d78beea53cdfe2b8815f2f7a929af7fd55ea70b33295453c5ed045cd64ca"}
Nov 02 22:00:05.459560 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.459030    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" podStartSLOduration=2.459006837 podStartE2EDuration="2.459006837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:05.457844763 +0000 UTC m=+1503.974340537" watchObservedRunningTime="2026-11-02 22:00:05.459006837 +0000 UTC m=+1503.975502610"
Nov 02 22:00:05.873888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.873854    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 22:00:05.874364 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.874341    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 22:00:05.875575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.875556    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 22:00:05.891952 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.891919    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 22:00:05.892604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.892578    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 22:00:05.893918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:05.893894    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 22:00:11.133167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.133126    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:00:11.138105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.138077    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.182733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.182692    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:00:11.187241 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.187215    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.234676 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.234633    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:00:11.270828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.270790    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:00:11.281634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.281584    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.281816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.281638    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.281816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.281778    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.281816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.281813    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282002    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282096    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6xw8\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282144    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282178    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282270 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282228    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282270 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282257    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282318    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282351    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282463 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282381    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282465    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282549    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.282628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282611    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282641    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6l7g\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.282682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.282666    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384187    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384232    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384378    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384468    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6xw8\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384507    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384535    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384583    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384598    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384615    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384681    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384718    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.384887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384753    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384838    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.384887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384868    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.384929    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385026    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385038    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385038    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385065    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6l7g\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.385207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385094    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.385715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385228    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.385843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385731    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.386354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385650    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.386354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.385455    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.386354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.386287    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.386354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.386313    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.386635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.386440    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.386694 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.386661    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.387516 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.387492    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.387625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.387614    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.387784 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.387762    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.388398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.388381    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.441322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.441285    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.451340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.441830    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6l7g\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\") pod \"maas-default-gateway-openshift-default-65c448b547-kq8f9\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.451340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.441906    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.451340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.442089    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6xw8\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\") pod \"maas-default-gateway-openshift-default-65c448b547-24z5d\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.455123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.455092    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:11.502046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.502005    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:11.885988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.885945    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:00:11.949688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.947502    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:00:11.951849 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.950660    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:11.951849 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.950761    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:11.951849 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.950803    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:11.959705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.959541    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:11.959705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.959634    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:11.959705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:11.959682    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:12.405664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.405624    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" event={"ID":"c790de74-3dd8-4fb4-b60b-91493b9b0dbc","Type":"ContainerStarted","Data":"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"}
Nov 02 22:00:12.406291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.406262    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" event={"ID":"c790de74-3dd8-4fb4-b60b-91493b9b0dbc","Type":"ContainerStarted","Data":"8492a4ffeb9eaabe0a5d385a37b59aa9c181534e9cfea688d47f1bef16d1ad5a"}
Nov 02 22:00:12.408892 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.408854    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" event={"ID":"4fea3fb8-425c-4000-8529-79f3f4ece0b0","Type":"ContainerStarted","Data":"495acdffc99b5dbf3bf1bb8cc321d543f0429aa8968afc127ee0ff39cb91c24c"}
Nov 02 22:00:12.409084 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.408921    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" event={"ID":"4fea3fb8-425c-4000-8529-79f3f4ece0b0","Type":"ContainerStarted","Data":"0cb1e4491ebbbe12bb0ae8c1e4cfa74974ec837a4453784765819dbbb0d51c93"}
Nov 02 22:00:12.469988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.468936    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:12.503317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.503282    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:12.555982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.555634    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podStartSLOduration=1.5556125779999999 podStartE2EDuration="1.555612578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:12.542483969 +0000 UTC m=+1511.058979846" watchObservedRunningTime="2026-11-02 22:00:12.555612578 +0000 UTC m=+1511.072108351"
Nov 02 22:00:12.677828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:12.677770    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podStartSLOduration=1.6777464210000002 podStartE2EDuration="1.677746421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:12.665650623 +0000 UTC m=+1511.182146394" watchObservedRunningTime="2026-11-02 22:00:12.677746421 +0000 UTC m=+1511.194242193"
Nov 02 22:00:13.462214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:13.462174    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:13.462778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:13.462747    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:13.508170 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:13.508136    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:13.508463 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:13.508428    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:14.455706 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:14.455671    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:14.455897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:14.455720    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:14.525598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:14.525559    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:14.526205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:14.526169    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:15.455593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:15.455548    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:15.455749 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:15.455631    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:15.514981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:15.509414    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:15.514981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:15.509511    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:16.354477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:16.354445    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:00:16.456035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:16.455995    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:16.456323 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:16.456295    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:16.532980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:16.508705    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:16.752601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:16.752549    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:17.457055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:17.456047    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:17.457055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:17.456096    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:17.506981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:17.506319    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:17.506981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:17.506410    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:18.455763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:18.455723    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:18.455952 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:18.455790    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:18.503508 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:18.503338    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:18.503508 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:18.503401    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:19.455825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:19.455792    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 02 22:00:19.456081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:19.455853    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 02 22:00:19.503227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:19.503188    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 02 22:00:19.503412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:19.503257    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 02 22:00:21.246977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.242620    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:21.246977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.243623    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:21.246977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.244748    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:00:21.328489 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.327244    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:21.328489 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.328430    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:21.331737 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.331667    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:00:21.411109 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.411077    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:00:21.415618 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.415592    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.466534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.458462    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.466534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.458639    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtpn8\" (UniqueName: \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.487979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.481758    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:00:21.487979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.484396    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-d1c62e-dockercfg-mz9l5\""
Nov 02 22:00:21.487979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.484520    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-d1c62e\""
Nov 02 22:00:21.505064 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.502128    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:00:21.519386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.519356    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.576312 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.576281    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-34089f5b-openshift-default-dockercfg-hzzd7\""
Nov 02 22:00:21.585572 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.579419    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:00:21.585899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.585867    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.588129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.588101    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.588425 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.588393    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gvj7\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.588589 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.588567    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.588764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.588745    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.588974 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.588940    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.589141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.589122    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtpn8\" (UniqueName: \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.590156 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.590133    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.590331 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.590313    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.590527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.590506    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.590736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.590717    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.591365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.591345    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.647754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.647719    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtpn8\" (UniqueName: \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\") pod \"payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.692643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692611    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.692817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692679    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.692817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692715    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.692817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692777    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.693075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692856    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.693075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.692894    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.693180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.693042    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gvj7\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.693224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.693199    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.693879 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.693604    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.694513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.694474    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.694833 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.694794    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.695313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.695163    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.695542 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.695520    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.704976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.696981    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.704976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.697759    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.719278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.719242    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.740346 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.740308    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:21.785354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.785065    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.785354 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.785296    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gvj7\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:21.840147 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:21.839820    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:22.181812 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.175969    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:00:22.288760 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:00:22.288723    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e871aa7_b054_494f_a1a6_e5ae0eebff30.slice/crio-20de06a51b52660ba55a8be9c94f6ea2e1be7de0f9aa60c5648a623975f17168 WatchSource:0}: Error finding container 20de06a51b52660ba55a8be9c94f6ea2e1be7de0f9aa60c5648a623975f17168: Status 404 returned error can't find the container with id 20de06a51b52660ba55a8be9c94f6ea2e1be7de0f9aa60c5648a623975f17168
Nov 02 22:00:22.289339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.289286    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:00:22.291841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.291801    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:22.292027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.291878    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:22.292027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.291910    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:00:22.851741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.851701    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" event={"ID":"2e871aa7-b054-494f-a1a6-e5ae0eebff30","Type":"ContainerStarted","Data":"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"}
Nov 02 22:00:22.851913 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.851770    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" event={"ID":"2e871aa7-b054-494f-a1a6-e5ae0eebff30","Type":"ContainerStarted","Data":"20de06a51b52660ba55a8be9c94f6ea2e1be7de0f9aa60c5648a623975f17168"}
Nov 02 22:00:22.853662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.853626    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" event={"ID":"d60eb4a7-af94-4e56-89d7-bf3202287774","Type":"ContainerStarted","Data":"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"}
Nov 02 22:00:22.853809 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.853687    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" event={"ID":"d60eb4a7-af94-4e56-89d7-bf3202287774","Type":"ContainerStarted","Data":"e7d273b38d784a8d0612d360a7491a7aecfcebdd80da98b191b37c0006c6566e"}
Nov 02 22:00:22.975766 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:22.975697    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podStartSLOduration=1.975675511 podStartE2EDuration="1.975675511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:22.970392986 +0000 UTC m=+1521.486888782" watchObservedRunningTime="2026-11-02 22:00:22.975675511 +0000 UTC m=+1521.492171284"
Nov 02 22:00:23.845597 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:23.845368    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:23.847884 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:23.847859    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:00:23.848058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:23.847944    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 02 22:00:23.861215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:23.861183    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:24.842641 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:24.842606    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:00:24.842842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:24.842662    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 02 22:00:25.842835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:25.842802    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:00:25.843250 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:25.842856    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 02 22:00:26.329163 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:26.329124    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:00:26.843345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:26.843303    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:00:26.843730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:26.843367    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 02 22:00:27.842610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:27.842567    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:00:27.842969 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:27.842627    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 02 22:00:28.847825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:28.847790    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:28.848341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:28.848228    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:28.849091 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:28.849068    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:00:29.016007 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:29.015900    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" podStartSLOduration=8.015879318 podStartE2EDuration="8.015879318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:23.090344004 +0000 UTC m=+1521.606839776" watchObservedRunningTime="2026-11-02 22:00:29.015879318 +0000 UTC m=+1527.532375091"
Nov 02 22:00:32.922839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:32.922800    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:00:32.928722 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:32.928696    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:32.980569 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:32.980537    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e2be7cc3-serving-cert\""
Nov 02 22:00:32.983583 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:32.983553    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:00:33.093680 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.093639    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.093862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.093823    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.094074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.094054    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdnpq\" (UniqueName: \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.094322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.094304    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.198614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.198504    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.198614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.198601    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.198889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.198759    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.198889 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:33.198864    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-serving-cert: secret "maas-api-e2e-mt-e2be7cc3-serving-cert" not found
Nov 02 22:00:33.200343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.199945    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.200343 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:33.200067    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls podName:ca2f7793-b888-4eed-bed4-71e8eee7c60d nodeName:}" failed. No retries permitted until 2026-11-02 22:00:33.69893686 +0000 UTC m=+1532.215432614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls") pod "maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" (UID: "ca2f7793-b888-4eed-bed4-71e8eee7c60d") : secret "maas-api-e2e-mt-e2be7cc3-serving-cert" not found
Nov 02 22:00:33.200343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.200221    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdnpq\" (UniqueName: \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.207223 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.207184    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.263323 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.263286    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdnpq\" (UniqueName: \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.716279 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.716235    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.718903 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.718874    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") pod \"maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:33.844916 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:33.844887    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:34.238243 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:34.228764    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:00:34.871573 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:34.871541    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:00:34.969507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:34.969473    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" event={"ID":"ca2f7793-b888-4eed-bed4-71e8eee7c60d","Type":"ContainerStarted","Data":"460c0b6cc5a4369bc2867fd0364ad0caad607c84b2df7f5dd1344382835c8902"}
Nov 02 22:00:35.979190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:35.979151    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" event={"ID":"ca2f7793-b888-4eed-bed4-71e8eee7c60d","Type":"ContainerStarted","Data":"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"}
Nov 02 22:00:35.979674 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:35.979295    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:36.113979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:36.102919    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" podStartSLOduration=3.187914981 podStartE2EDuration="4.102897287s" totalImagesPullingTime="914.982306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:00:36.099993517 +0000 UTC m=+1534.616489289" watchObservedRunningTime="2026-11-02 22:00:36.102897287 +0000 UTC m=+1534.619393061"
Nov 02 22:00:37.388130 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:37.388076    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 22:00:37.494767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:37.494732    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 22:00:41.992410 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:41.992331    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:00:42.389242 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.389119    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" podUID="170b7e7b-1efd-482b-9739-b486c80b7d32" containerName="payload-processing" containerID="cri-o://5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a" gracePeriod=25
Nov 02 22:00:42.495156 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.495093    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" podUID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" containerName="payload-pre-processing" containerID="cri-o://4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53" gracePeriod=25
Nov 02 22:00:42.715616 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.715583    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 22:00:42.795943 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.795916    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 22:00:42.811017 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.810985    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\") pod \"170b7e7b-1efd-482b-9739-b486c80b7d32\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") "
Nov 02 22:00:42.811239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.811222    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\" (UniqueName: \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\") pod \"170b7e7b-1efd-482b-9739-b486c80b7d32\" (UID: \"170b7e7b-1efd-482b-9739-b486c80b7d32\") "
Nov 02 22:00:42.811332 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.811311    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume" pod "170b7e7b-1efd-482b-9739-b486c80b7d32" (UID: "170b7e7b-1efd-482b-9739-b486c80b7d32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:42.811904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.811885    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/170b7e7b-1efd-482b-9739-b486c80b7d32-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:00:42.813365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.813342    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8" pod "170b7e7b-1efd-482b-9739-b486c80b7d32" (UID: "170b7e7b-1efd-482b-9739-b486c80b7d32"). InnerVolumeSpecName "kube-api-access-bnhh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:42.914187 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.914154    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\") pod \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") "
Nov 02 22:00:42.914368 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.914261    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\" (UniqueName: \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\") pod \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\" (UID: \"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe\") "
Nov 02 22:00:42.914607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.914580    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume" pod "73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" (UID: "73d4bcbb-fa2a-4735-97e2-71e4b9436dbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:00:42.914907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.914889    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:00:42.914907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.914908    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnhh8\" (UniqueName: \"kubernetes.io/projected/170b7e7b-1efd-482b-9739-b486c80b7d32-kube-api-access-bnhh8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:00:42.916590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:42.916566    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j" pod "73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" (UID: "73d4bcbb-fa2a-4735-97e2-71e4b9436dbe"). InnerVolumeSpecName "kube-api-access-5mw7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:00:43.016794 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.016704    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mw7j\" (UniqueName: \"kubernetes.io/projected/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe-kube-api-access-5mw7j\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:00:43.071686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.071650    5580 generic.go:356] "Generic (PLEG): container finished" podID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" containerID="4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53" exitCode=0
Nov 02 22:00:43.071900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.071692    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"
Nov 02 22:00:43.071900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.071734    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" event={"ID":"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe","Type":"ContainerDied","Data":"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"}
Nov 02 22:00:43.071900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.071826    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h" event={"ID":"73d4bcbb-fa2a-4735-97e2-71e4b9436dbe","Type":"ContainerDied","Data":"9342a2ab0aab842e57e5374f7d74053605b1a6302ad798e03b506a05080cd9f0"}
Nov 02 22:00:43.071900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.071857    5580 scope.go:122] "RemoveContainer" containerID="4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"
Nov 02 22:00:43.073523 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.073499    5580 generic.go:356] "Generic (PLEG): container finished" podID="170b7e7b-1efd-482b-9739-b486c80b7d32" containerID="5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a" exitCode=0
Nov 02 22:00:43.073634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.073547    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" event={"ID":"170b7e7b-1efd-482b-9739-b486c80b7d32","Type":"ContainerDied","Data":"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"}
Nov 02 22:00:43.073634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.073550    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"
Nov 02 22:00:43.073634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.073584    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7" event={"ID":"170b7e7b-1efd-482b-9739-b486c80b7d32","Type":"ContainerDied","Data":"1eacba0be03d7c215960336e1c2c0f8925dd0f1b2cf542a509436c198ad68473"}
Nov 02 22:00:43.083841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.083816    5580 scope.go:122] "RemoveContainer" containerID="4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"
Nov 02 22:00:43.084217 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:43.084190    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53\": container with ID starting with 4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53 not found: ID does not exist" containerID="4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"
Nov 02 22:00:43.084319 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.084224    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53"} err="failed to get container status \"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53\": rpc error: code = NotFound desc = could not find container \"4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53\": container with ID starting with 4598fac79a3f90ad0ba43d253980aa4f235366ded1ba09c8c4f754a47de47c53 not found: ID does not exist"
Nov 02 22:00:43.084319 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.084251    5580 scope.go:122] "RemoveContainer" containerID="5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"
Nov 02 22:00:43.095128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.095106    5580 scope.go:122] "RemoveContainer" containerID="5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"
Nov 02 22:00:43.095425 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:43.095405    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a\": container with ID starting with 5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a not found: ID does not exist" containerID="5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"
Nov 02 22:00:43.095470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.095436    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a"} err="failed to get container status \"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a\": rpc error: code = NotFound desc = could not find container \"5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a\": container with ID starting with 5f5bc85e7c03c31c2411699cad36f2a22adcaeb6c02a5dfcabd8fc0612398c8a not found: ID does not exist"
Nov 02 22:00:43.236754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.236706    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 22:00:43.241805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.241769    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-34089f5b-6b779ddc-jlg5h"]
Nov 02 22:00:43.348352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.348221    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 22:00:43.366980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.354601    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34089f5b-5c47c5949c-gm2k7"]
Nov 02 22:00:43.597079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.597042    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="170b7e7b-1efd-482b-9739-b486c80b7d32" path="/var/lib/kubelet/pods/170b7e7b-1efd-482b-9739-b486c80b7d32/volumes"
Nov 02 22:00:43.597622 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:43.597607    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" path="/var/lib/kubelet/pods/73d4bcbb-fa2a-4735-97e2-71e4b9436dbe/volumes"
Nov 02 22:00:46.565755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:00:46.565714    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:00:56.312238 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:00:56.312199    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:01:06.818268 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.818225    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:06.819021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819003    5580 cpu_manager.go:414] "containerMap: removing container" podUID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" containerName="payload-pre-processing"
Nov 02 22:01:06.819021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819021    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" containerName="payload-pre-processing"
Nov 02 22:01:06.819125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819041    5580 cpu_manager.go:414] "containerMap: removing container" podUID="170b7e7b-1efd-482b-9739-b486c80b7d32" containerName="payload-processing"
Nov 02 22:01:06.819125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819047    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="170b7e7b-1efd-482b-9739-b486c80b7d32" containerName="payload-processing"
Nov 02 22:01:06.819196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819186    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="73d4bcbb-fa2a-4735-97e2-71e4b9436dbe" containerName="payload-pre-processing"
Nov 02 22:01:06.819237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.819199    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="170b7e7b-1efd-482b-9739-b486c80b7d32" containerName="payload-processing"
Nov 02 22:01:06.825022 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.824996    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.892862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.892824    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.892869    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6h99\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.892896    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893066    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893200    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893349 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893307    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893412    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893473    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.893538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.893496    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:06.930721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.930675    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.995670    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6h99\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998029    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998122    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998252    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998352    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998492    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998572    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998599    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.998784    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.999344    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.999568    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.000756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:06.999801    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.026984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.004417    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.026984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.018316    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.035034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.032737    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.035034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.033162    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.054552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.054511    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6h99\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.058345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.058307    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\") pod \"e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") " pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.143330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.143239    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:07.659385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.645574    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:07.659385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.647565    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:01:07.659385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.649192    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:01:07.659385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:07.649677    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:01:08.318341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:08.318302    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" event={"ID":"8e3cfd67-9f71-4dbd-9412-fef9ec30b799","Type":"ContainerStarted","Data":"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"}
Nov 02 22:01:08.318775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:08.318372    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" event={"ID":"8e3cfd67-9f71-4dbd-9412-fef9ec30b799","Type":"ContainerStarted","Data":"873c607173a7bfc7f1358e61445614c4fc8c25ad8e8a923d772e285f82b1b37c"}
Nov 02 22:01:08.436121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:08.436063    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podStartSLOduration=2.436046023 podStartE2EDuration="2.436046023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:01:08.434309451 +0000 UTC m=+1566.950805248" watchObservedRunningTime="2026-11-02 22:01:08.436046023 +0000 UTC m=+1566.952541867"
Nov 02 22:01:09.144040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:09.143995    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:09.145585 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:09.145560    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:01:09.145720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:09.145609    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 02 22:01:10.144246 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:10.144209    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:01:10.144655 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:10.144267    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 02 22:01:11.144082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:11.144038    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:01:11.144274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:11.144117    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 02 22:01:12.143975 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:12.143918    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:01:12.144186 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:12.144050    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 02 22:01:13.146015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.144211    5580 patch_prober.go:29] interesting pod/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:01:13.146015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.144274    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 02 22:01:13.861171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.861133    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:01:13.862164 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.861359    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy" containerID="cri-o://2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b" gracePeriod=30
Nov 02 22:01:13.874685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.874646    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:13.874939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:13.874891    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy" containerID="cri-o://c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e" gracePeriod=30
Nov 02 22:01:19.158195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.158164    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:01:19.288804 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.288766    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.288969    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289006    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289074    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289099    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289170    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289355    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289416    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289453    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\") pod \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\" (UID: \"2e871aa7-b054-494f-a1a6-e5ae0eebff30\") "
Nov 02 22:01:19.289994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.289620    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.291204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.290333    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.291204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.290569    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.291204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.290748    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.291204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.290806    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.291702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.291665    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:01:19.294786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.293320    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.294786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.294116    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:01:19.295864 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.295829    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:01:19.296460 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.296427    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7" pod "2e871aa7-b054-494f-a1a6-e5ae0eebff30" (UID: "2e871aa7-b054-494f-a1a6-e5ae0eebff30"). InnerVolumeSpecName "kube-api-access-4gvj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:01:19.353191 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.353155    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392178    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392206    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392219    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392228    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gvj7\" (UniqueName: \"kubernetes.io/projected/2e871aa7-b054-494f-a1a6-e5ae0eebff30-kube-api-access-4gvj7\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392240    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392249    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392259    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e871aa7-b054-494f-a1a6-e5ae0eebff30-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.392260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.392267    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e871aa7-b054-494f-a1a6-e5ae0eebff30-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.467468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.467428    5580 generic.go:356] "Generic (PLEG): container finished" podID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerID="2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b" exitCode=0
Nov 02 22:01:19.467468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.467476    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"
Nov 02 22:01:19.467764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.467521    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" event={"ID":"2e871aa7-b054-494f-a1a6-e5ae0eebff30","Type":"ContainerDied","Data":"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"}
Nov 02 22:01:19.467764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.467599    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv" event={"ID":"2e871aa7-b054-494f-a1a6-e5ae0eebff30","Type":"ContainerDied","Data":"20de06a51b52660ba55a8be9c94f6ea2e1be7de0f9aa60c5648a623975f17168"}
Nov 02 22:01:19.467764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.467630    5580 scope.go:122] "RemoveContainer" containerID="2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"
Nov 02 22:01:19.481642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.481603    5580 generic.go:356] "Generic (PLEG): container finished" podID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerID="c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e" exitCode=0
Nov 02 22:01:19.481821 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.481803    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" event={"ID":"8e3cfd67-9f71-4dbd-9412-fef9ec30b799","Type":"ContainerDied","Data":"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"}
Nov 02 22:01:19.481914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.481901    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p" event={"ID":"8e3cfd67-9f71-4dbd-9412-fef9ec30b799","Type":"ContainerDied","Data":"873c607173a7bfc7f1358e61445614c4fc8c25ad8e8a923d772e285f82b1b37c"}
Nov 02 22:01:19.482051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.482035    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"
Nov 02 22:01:19.486206 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.486171    5580 scope.go:122] "RemoveContainer" containerID="2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"
Nov 02 22:01:19.494598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494563    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494647    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494696    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494840 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494767    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494840 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494814    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494865    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.494991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494926    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.495037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.494999    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.495143 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.495125    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\") pod \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\" (UID: \"8e3cfd67-9f71-4dbd-9412-fef9ec30b799\") "
Nov 02 22:01:19.500236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.500202    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:01:19.500623 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.500600    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.501437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.501407    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.501641 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.501613    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.501988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.501944    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.502651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.502622    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:01:19.503361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.503333    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:01:19.506484 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:01:19.506187    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b\": container with ID starting with 2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b not found: ID does not exist" containerID="2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"
Nov 02 22:01:19.506484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.506234    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b"} err="failed to get container status \"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b\": rpc error: code = NotFound desc = could not find container \"2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b\": container with ID starting with 2b453ceb5400b1d308c27189de47db55a967905d97a47bff152aabdd409b005b not found: ID does not exist"
Nov 02 22:01:19.506484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.506263    5580 scope.go:122] "RemoveContainer" containerID="c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"
Nov 02 22:01:19.507730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.507679    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:01:19.509424 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.509396    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99" pod "8e3cfd67-9f71-4dbd-9412-fef9ec30b799" (UID: "8e3cfd67-9f71-4dbd-9412-fef9ec30b799"). InnerVolumeSpecName "kube-api-access-b6h99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:01:19.519460 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.519419    5580 scope.go:122] "RemoveContainer" containerID="c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"
Nov 02 22:01:19.519938 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:01:19.519903    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e\": container with ID starting with c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e not found: ID does not exist" containerID="c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"
Nov 02 22:01:19.520072 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.520020    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e"} err="failed to get container status \"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e\": rpc error: code = NotFound desc = could not find container \"c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e\": container with ID starting with c0daee0f2b0b2c3a8774e45a0836533ba28108d28baabbec82f23a6ef62a982e not found: ID does not exist"
Nov 02 22:01:19.597750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597720    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6h99\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-kube-api-access-b6h99\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.597750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597748    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.597750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597758    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597767    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597776    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597785    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597793    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597802    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.598020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.597810    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e3cfd67-9f71-4dbd-9412-fef9ec30b799-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:19.720685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.720647    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:01:19.724690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.724653    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-flvnv"]
Nov 02 22:01:19.897586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.897559    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:19.902998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:19.902948    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-34089f5b-openshift-default-68c7cb699f-j6g2p"]
Nov 02 22:01:21.596728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:21.596682    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" path="/var/lib/kubelet/pods/2e871aa7-b054-494f-a1a6-e5ae0eebff30/volumes"
Nov 02 22:01:21.597400 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:21.597383    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" path="/var/lib/kubelet/pods/8e3cfd67-9f71-4dbd-9412-fef9ec30b799/volumes"
Nov 02 22:01:22.896571 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:22.896528    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 22:01:22.897140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:22.896825    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" podUID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" containerName="maas-api" containerID="cri-o://d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd" gracePeriod=30
Nov 02 22:01:23.231867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.231833    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 22:01:23.371051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.371013    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\") pod \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") "
Nov 02 22:01:23.371398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.371379    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") pod \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") "
Nov 02 22:01:23.371994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.371555    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\" (UniqueName: \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\") pod \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") "
Nov 02 22:01:23.372208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.371443    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca" pod "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" (UID: "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:01:23.372321 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.372292    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\") pod \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\" (UID: \"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0\") "
Nov 02 22:01:23.373268 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.373244    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:23.376010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.374726    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls" pod "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" (UID: "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:01:23.379520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.379486    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs" pod "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" (UID: "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:01:23.380195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.380122    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5" pod "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" (UID: "5873d9d3-7d77-4f8a-b024-eab0d8fa62d0"). InnerVolumeSpecName "kube-api-access-67qv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:01:23.475451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.475338    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:23.475451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.475385    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67qv5\" (UniqueName: \"kubernetes.io/projected/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-kube-api-access-67qv5\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:23.475451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.475402    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:01:23.517901 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.517865    5580 generic.go:356] "Generic (PLEG): container finished" podID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" containerID="d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd" exitCode=0
Nov 02 22:01:23.518120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.517917    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"
Nov 02 22:01:23.518120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.517947    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" event={"ID":"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0","Type":"ContainerDied","Data":"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"}
Nov 02 22:01:23.518120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.518043    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx" event={"ID":"5873d9d3-7d77-4f8a-b024-eab0d8fa62d0","Type":"ContainerDied","Data":"da21fdbc2886baa3377ac438f6d80aac8644bca8a7d44286b42e9087322c7fb4"}
Nov 02 22:01:23.518120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.518066    5580 scope.go:122] "RemoveContainer" containerID="d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"
Nov 02 22:01:23.529763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.529742    5580 scope.go:122] "RemoveContainer" containerID="d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"
Nov 02 22:01:23.530121 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:01:23.530100    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd\": container with ID starting with d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd not found: ID does not exist" containerID="d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"
Nov 02 22:01:23.530179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.530133    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd"} err="failed to get container status \"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd\": rpc error: code = NotFound desc = could not find container \"d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd\": container with ID starting with d73083a3a98e1d856312645c4bb273c22e1b35491e23ff148dca26e7b0a774dd not found: ID does not exist"
Nov 02 22:01:23.697112 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.697080    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 22:01:23.701644 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:23.701610    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a4e90e53-6877d795b9-zw9rx"]
Nov 02 22:01:25.595179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:25.595145    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" path="/var/lib/kubelet/pods/5873d9d3-7d77-4f8a-b024-eab0d8fa62d0/volumes"
Nov 02 22:01:26.313028 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:01:26.312948    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:01:36.751229 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:01:36.751184    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:01:56.314898 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:01:56.314851    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:02:15.311477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:15.311255    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:02:20.270258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:20.270201    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:02:20.371163 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:20.371076    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 22:02:20.479486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:20.479419    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:02:22.933903 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.933788    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:02:22.935690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.935657    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" containerName="maas-api"
Nov 02 22:02:22.935879 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.935863    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" containerName="maas-api"
Nov 02 22:02:22.936023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936006    5580 cpu_manager.go:414] "containerMap: removing container" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy"
Nov 02 22:02:22.936111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936082    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy"
Nov 02 22:02:22.936111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936107    5580 cpu_manager.go:414] "containerMap: removing container" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy"
Nov 02 22:02:22.936209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936118    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy"
Nov 02 22:02:22.936340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936324    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5873d9d3-7d77-4f8a-b024-eab0d8fa62d0" containerName="maas-api"
Nov 02 22:02:22.936399 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936346    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e3cfd67-9f71-4dbd-9412-fef9ec30b799" containerName="istio-proxy"
Nov 02 22:02:22.936399 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.936358    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e871aa7-b054-494f-a1a6-e5ae0eebff30" containerName="istio-proxy"
Nov 02 22:02:22.940409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.940372    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:22.973942 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.973895    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4qb\" (UniqueName: \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:22.974238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.974134    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:22.987395 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:22.987338    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:02:23.024665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.024621    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4fc235\""
Nov 02 22:02:23.024874 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.024746    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4fc235-dockercfg-n4q47\""
Nov 02 22:02:23.075615 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.075570    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4qb\" (UniqueName: \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:23.075845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.075659    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:23.076439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.076412    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:23.132634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.132596    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4qb\" (UniqueName: \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\") pod \"payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") " pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:23.287384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.287282    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:23.775860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:23.775820    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:02:23.777132 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:02:23.777101    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2408876_6604_456f_bdb1_b939c1945905.slice/crio-9f8e21e0343f8dc5ed2f4c615274754fb920d82dfc78e32f5eee004d16b20a5e WatchSource:0}: Error finding container 9f8e21e0343f8dc5ed2f4c615274754fb920d82dfc78e32f5eee004d16b20a5e: Status 404 returned error can't find the container with id 9f8e21e0343f8dc5ed2f4c615274754fb920d82dfc78e32f5eee004d16b20a5e
Nov 02 22:02:24.433874 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:24.433831    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" event={"ID":"a2408876-6604-456f-bdb1-b939c1945905","Type":"ContainerStarted","Data":"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"}
Nov 02 22:02:24.434361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:24.433904    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:24.434361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:24.433927    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" event={"ID":"a2408876-6604-456f-bdb1-b939c1945905","Type":"ContainerStarted","Data":"9f8e21e0343f8dc5ed2f4c615274754fb920d82dfc78e32f5eee004d16b20a5e"}
Nov 02 22:02:24.548598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:24.548544    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" podStartSLOduration=2.548527823 podStartE2EDuration="2.548527823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:02:24.546229521 +0000 UTC m=+1643.062725330" watchObservedRunningTime="2026-11-02 22:02:24.548527823 +0000 UTC m=+1643.065023803"
Nov 02 22:02:26.312876 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:26.312835    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:02:32.626605 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.626569    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:02:32.631925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.631870    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.693822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.693780    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:02:32.736239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.736201    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-2a9be7-dockercfg-99lpn\""
Nov 02 22:02:32.736504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.736313    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-2a9be7\""
Nov 02 22:02:32.796406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.796353    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.796715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.796685    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm4pr\" (UniqueName: \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.900979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.900857    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.901284 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.901258    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm4pr\" (UniqueName: \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.902278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.902241    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:32.958418 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:32.958382    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm4pr\" (UniqueName: \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\") pod \"payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:33.256141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:33.256102    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:33.639342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:33.639301    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:02:33.639702 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:02:33.639424    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610477ee_b768_4bd3_8dfc_e4e062145374.slice/crio-9bc25fe7d0f3430e0a485200e438f760067b0e3f756de021d6efd42f702f5100 WatchSource:0}: Error finding container 9bc25fe7d0f3430e0a485200e438f760067b0e3f756de021d6efd42f702f5100: Status 404 returned error can't find the container with id 9bc25fe7d0f3430e0a485200e438f760067b0e3f756de021d6efd42f702f5100
Nov 02 22:02:34.524233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:34.524190    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" event={"ID":"610477ee-b768-4bd3-8dfc-e4e062145374","Type":"ContainerStarted","Data":"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"}
Nov 02 22:02:34.524417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:34.524250    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" event={"ID":"610477ee-b768-4bd3-8dfc-e4e062145374","Type":"ContainerStarted","Data":"9bc25fe7d0f3430e0a485200e438f760067b0e3f756de021d6efd42f702f5100"}
Nov 02 22:02:34.524417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:34.524272    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:34.639316 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:34.639248    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" podStartSLOduration=2.639231932 podStartE2EDuration="2.639231932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:02:34.636264393 +0000 UTC m=+1653.152760157" watchObservedRunningTime="2026-11-02 22:02:34.639231932 +0000 UTC m=+1653.155727704"
Nov 02 22:02:35.445198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:35.445167    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:02:45.278118 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.278083    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 22:02:45.278529 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.278297    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy" containerID="cri-o://cfca3b9b3e5b488f39a607e5bb2d082342b63796dd9289ccc01af81c478dd27f" gracePeriod=30
Nov 02 22:02:45.327624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.327584    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 22:02:45.327879 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.327852    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy" containerID="cri-o://fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818" gracePeriod=30
Nov 02 22:02:45.349310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.349264    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 22:02:45.349597 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.349562    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" containerID="cri-o://551deb0c4c65134f5e7a298f43af60f480f874f737bf9c689959fd2e1768eef0" gracePeriod=30
Nov 02 22:02:45.377851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.377812    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 22:02:45.378096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.378069    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" containerID="cri-o://69fa266b24e998b8e284b6d95bc65cf1edaf40e34e2f7be59898804ad9f906fc" gracePeriod=30
Nov 02 22:02:45.535389 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.535293    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:02:45.661680 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.661637    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 22:02:45.661932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.661904    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy" containerID="cri-o://9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787" gracePeriod=30
Nov 02 22:02:45.665862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.665821    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 22:02:45.666265 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.666217    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy" containerID="cri-o://633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479" gracePeriod=30
Nov 02 22:02:45.670767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.670726    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 22:02:45.671007 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.670978    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy" containerID="cri-o://ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110" gracePeriod=30
Nov 02 22:02:45.676564 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.676526    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 22:02:45.676815 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:45.676784    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" containerID="cri-o://051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193" gracePeriod=30
Nov 02 22:02:47.338048 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.338011    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 22:02:47.338628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.338240    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" containerID="cri-o://325e5e8bdbb5285b2e00ce0cd849d7f32629eda6d2880f44c1a334aa46e4e82f" gracePeriod=30
Nov 02 22:02:47.342412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.342378    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 22:02:47.342650 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.342624    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" containerID="cri-o://3e2938c05fdc1fac8f15d43d72c271067d420af7e76bbacdd1d6cd29ba8b000c" gracePeriod=30
Nov 02 22:02:47.392010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.391950    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 22:02:47.392214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.392186    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy" containerID="cri-o://2646d78b93c375cfc198f8ef84cd58b573d8147c042baeb46b2591d62a6f6ef0" gracePeriod=30
Nov 02 22:02:47.397477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.397439    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 22:02:47.397797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.397765    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy" containerID="cri-o://45ce881187d21992e88f9f5da8e9c8b81177906e63d50b484e7a1ffcca01da6c" gracePeriod=30
Nov 02 22:02:47.432125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.432086    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:02:47.432303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:47.432157    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:02:48.233532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:48.233491    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:02:48.233728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:48.233556    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:02:50.166750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.166714    5580 patch_prober.go:29] interesting pod/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:02:50.167178 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.166784    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:02:50.219531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.219493    5580 patch_prober.go:29] interesting pod/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:02:50.219721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.219551    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:02:50.637129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.636518    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 22:02:50.675843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.675752    5580 generic.go:356] "Generic (PLEG): container finished" podID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerID="fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818" exitCode=0
Nov 02 22:02:50.676335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.675825    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"
Nov 02 22:02:50.678437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.675847    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" event={"ID":"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f","Type":"ContainerDied","Data":"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"}
Nov 02 22:02:50.678437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.678042    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f" event={"ID":"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f","Type":"ContainerDied","Data":"9418aad11fe3fab5584d4d54c4299520d271ecac9720dbfd43cb1c52aa9bcb53"}
Nov 02 22:02:50.678437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.678084    5580 scope.go:122] "RemoveContainer" containerID="fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"
Nov 02 22:02:50.682328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.682061    5580 generic.go:356] "Generic (PLEG): container finished" podID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerID="551deb0c4c65134f5e7a298f43af60f480f874f737bf9c689959fd2e1768eef0" exitCode=0
Nov 02 22:02:50.682328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.682250    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" event={"ID":"d1fb38fa-c786-4e2c-b929-b03690d8a717","Type":"ContainerDied","Data":"551deb0c4c65134f5e7a298f43af60f480f874f737bf9c689959fd2e1768eef0"}
Nov 02 22:02:50.684854 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.684658    5580 generic.go:356] "Generic (PLEG): container finished" podID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerID="cfca3b9b3e5b488f39a607e5bb2d082342b63796dd9289ccc01af81c478dd27f" exitCode=0
Nov 02 22:02:50.684854 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.684772    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" event={"ID":"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c","Type":"ContainerDied","Data":"cfca3b9b3e5b488f39a607e5bb2d082342b63796dd9289ccc01af81c478dd27f"}
Nov 02 22:02:50.687641 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.687606    5580 generic.go:356] "Generic (PLEG): container finished" podID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerID="69fa266b24e998b8e284b6d95bc65cf1edaf40e34e2f7be59898804ad9f906fc" exitCode=0
Nov 02 22:02:50.687761 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.687676    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" event={"ID":"2e1ace86-270d-4edc-a433-689ce394e8d0","Type":"ContainerDied","Data":"69fa266b24e998b8e284b6d95bc65cf1edaf40e34e2f7be59898804ad9f906fc"}
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.836738    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.836800    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.836849    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.836888    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.837015    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.837059    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.837088    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.837326    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.838685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.837567    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\") pod \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\" (UID: \"4c68e7e0-c1f4-4df9-af1a-04c46b3be70f\") "
Nov 02 22:02:50.839702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.839677    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:50.839929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.839906    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:50.843348 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.841066    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:50.844115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.843720    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:50.844318 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.844276    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:50.852712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.852643    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:50.854125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.853661    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "kube-api-access-4vkww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:50.855093 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.855067    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:50.855192 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.855161    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token" pod "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" (UID: "4c68e7e0-c1f4-4df9-af1a-04c46b3be70f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:50.882027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.879624    5580 scope.go:122] "RemoveContainer" containerID="fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"
Nov 02 22:02:50.882027 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:50.880823    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818\": container with ID starting with fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818 not found: ID does not exist" containerID="fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"
Nov 02 22:02:50.882027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.880864    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818"} err="failed to get container status \"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818\": rpc error: code = NotFound desc = could not find container \"fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818\": container with ID starting with fb6948816ae68b36c3b0946aae55a600b4fc23fdb854288ab5f7480ec312c818 not found: ID does not exist"
Nov 02 22:02:50.909195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.909163    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 22:02:50.913407 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.913159    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 22:02:50.921237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.921134    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941228    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941253    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941263    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941272    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941312    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941313 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941321    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941330    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941338    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:50.941767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:50.941346    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vkww\" (UniqueName: \"kubernetes.io/projected/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f-kube-api-access-4vkww\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.021341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.021314    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 22:02:51.042118 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.042078    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.042314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.042161    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.042748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.042719    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.042997 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.042938    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.043237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.042222    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043287    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.043362 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043337    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043392    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.043471 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043454    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043492    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043569 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043556    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.043617 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043586    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.043665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043638    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.043717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043680    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.043772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043718    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043820 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043787    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.043879 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043845    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.043927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043896    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.044056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043922    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.044056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.043951    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.044056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044011    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.044224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044066    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.044224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044130    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.044319 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044278    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.044376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044358    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\") pod \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\" (UID: \"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c\") "
Nov 02 22:02:51.044432 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044416    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.044503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044487    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.044635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044617    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\") pod \"d1fb38fa-c786-4e2c-b929-b03690d8a717\" (UID: \"d1fb38fa-c786-4e2c-b929-b03690d8a717\") "
Nov 02 22:02:51.044853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.044836    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\") pod \"2e1ace86-270d-4edc-a433-689ce394e8d0\" (UID: \"2e1ace86-270d-4edc-a433-689ce394e8d0\") "
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.045597    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "kube-api-access-qvgcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.046224    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvgcm\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-kube-api-access-qvgcm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.046248    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.046284    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.046805    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.046897    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047110    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047129    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047312    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047320    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047495    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047527    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.047789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047702    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.048546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.047900    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.048546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.048097    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.048681 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.048658    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.048855 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.048824    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.050170 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.050083    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.051113 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.050314    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.051113 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.050920    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "kube-api-access-kp2mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.051482 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.051143    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.051482 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.051451    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.053786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.053305    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy" pod "2e1ace86-270d-4edc-a433-689ce394e8d0" (UID: "2e1ace86-270d-4edc-a433-689ce394e8d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.053786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.053698    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.053786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.053732    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.053987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.053856    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.053987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.053904    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt" pod "d1fb38fa-c786-4e2c-b929-b03690d8a717" (UID: "d1fb38fa-c786-4e2c-b929-b03690d8a717"). InnerVolumeSpecName "kube-api-access-xm7xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.055585 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.055542    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo" pod "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" (UID: "81fd7f81-3fc6-4fe6-8d1c-77a7e842313c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.083928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.083841    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 22:02:51.130394 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.130363    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 22:02:51.143691 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.143436    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 22:02:51.148102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148051    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148285 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148148    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148285 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148232    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148401 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148374    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148413    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148445    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148537 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148494    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148615    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.148921 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.148676    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\") pod \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\" (UID: \"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5\") "
Nov 02 22:02:51.149475 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149439    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149475 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149466    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149481    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149492    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149506    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149519    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp2mv\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-kube-api-access-kp2mv\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149533    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm7xt\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-kube-api-access-xm7xt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149542    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149559    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e1ace86-270d-4edc-a433-689ce394e8d0-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149570    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149585    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149595    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149605    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149613    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149622    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.149634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149635    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149648    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1fb38fa-c786-4e2c-b929-b03690d8a717-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149663    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149675    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149706    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e1ace86-270d-4edc-a433-689ce394e8d0-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149811    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1fb38fa-c786-4e2c-b929-b03690d8a717-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149824    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1fb38fa-c786-4e2c-b929-b03690d8a717-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149835    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.149849    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e1ace86-270d-4edc-a433-689ce394e8d0-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.150379 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.150102    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.154677 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.153198    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.154677 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.153427    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.173982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.161196    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.173982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.163111    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.173982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.165281    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "kube-api-access-8kpww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.173982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.173224    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.173982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.173473    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.175557 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.175521    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy" pod "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" (UID: "ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.251365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251268    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.251365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251309    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.251365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251354    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251399    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251435    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251472    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251516    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251566    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.251667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251617    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.251875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251668    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.251875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251650    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.251875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251685    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.251875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251823    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.252034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251918    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.252034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.251944    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252040    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252079    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.252139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252120    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252159    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252202    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252241    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252246    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252256    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.252294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252278    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252322    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252358    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252396    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252430    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252436    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252542    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252606    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252682    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252725    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252742    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252787    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\") pod \"a574a4f3-284a-49cd-ae37-27313ca6c421\" (UID: \"a574a4f3-284a-49cd-ae37-27313ca6c421\") "
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252852    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\") pod \"85d26d6f-037d-4880-ad82-5ac26e61ff20\" (UID: \"85d26d6f-037d-4880-ad82-5ac26e61ff20\") "
Nov 02 22:02:51.252883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252883    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.253257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252923    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\") pod \"617b5749-2132-4151-b919-aa96a10621ba\" (UID: \"617b5749-2132-4151-b919-aa96a10621ba\") "
Nov 02 22:02:51.253257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.252929    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.253842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.253817    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269455    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269488    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269503    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269517    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269531    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269546    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269560    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269574    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kpww\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-kube-api-access-8kpww\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269590    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d26d6f-037d-4880-ad82-5ac26e61ff20-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269604    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.254632    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.269656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269618    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.281906 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269678    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a574a4f3-284a-49cd-ae37-27313ca6c421-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.281918    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.281942    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.281986    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282003    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282016    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282028    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282041    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282053    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.254711    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.255385    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.269076    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.282126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.276524    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.282909 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.282879    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.288804 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.288153    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.288804 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.288174    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "kube-api-access-hr2mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.288804 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.288214    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "kube-api-access-jhpnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.289617 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.289557    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "kube-api-access-t22wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.293586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.293548    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.293803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.293768    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:51.294109 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.294069    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy" pod "a574a4f3-284a-49cd-ae37-27313ca6c421" (UID: "a574a4f3-284a-49cd-ae37-27313ca6c421"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.294400 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.294377    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo" pod "617b5749-2132-4151-b919-aa96a10621ba" (UID: "617b5749-2132-4151-b919-aa96a10621ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:51.300586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.300532    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy" pod "85d26d6f-037d-4880-ad82-5ac26e61ff20" (UID: "85d26d6f-037d-4880-ad82-5ac26e61ff20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383516    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617b5749-2132-4151-b919-aa96a10621ba-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383556    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383570    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383587    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383603    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr2mq\" (UniqueName: \"kubernetes.io/projected/617b5749-2132-4151-b919-aa96a10621ba-kube-api-access-hr2mq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383617    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.383555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383630    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383646    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhpnl\" (UniqueName: \"kubernetes.io/projected/85d26d6f-037d-4880-ad82-5ac26e61ff20-kube-api-access-jhpnl\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.383659    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384307    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384334    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t22wm\" (UniqueName: \"kubernetes.io/projected/a574a4f3-284a-49cd-ae37-27313ca6c421-kube-api-access-t22wm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384350    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a574a4f3-284a-49cd-ae37-27313ca6c421-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384366    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d26d6f-037d-4880-ad82-5ac26e61ff20-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384380    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617b5749-2132-4151-b919-aa96a10621ba-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.385338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.384395    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617b5749-2132-4151-b919-aa96a10621ba-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:51.387750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.387722    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 22:02:51.391298 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.391268    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-4bw2f"]
Nov 02 22:02:51.596245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.596153    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" path="/var/lib/kubelet/pods/4c68e7e0-c1f4-4df9-af1a-04c46b3be70f/volumes"
Nov 02 22:02:51.699040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.699000    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf" event={"ID":"81fd7f81-3fc6-4fe6-8d1c-77a7e842313c","Type":"ContainerDied","Data":"dff828b41acdf695197cca3be4e28f17db70de849f7bb0d00e067e28dcd34f57"}
Nov 02 22:02:51.699261 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.699045    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"
Nov 02 22:02:51.699261 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.699082    5580 scope.go:122] "RemoveContainer" containerID="cfca3b9b3e5b488f39a607e5bb2d082342b63796dd9289ccc01af81c478dd27f"
Nov 02 22:02:51.702265 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.701538    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg" event={"ID":"2e1ace86-270d-4edc-a433-689ce394e8d0","Type":"ContainerDied","Data":"9d5f31c757095e6d141d4919962528b17bd6469bd46d197740af9a7180467d71"}
Nov 02 22:02:51.702265 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.701555    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"
Nov 02 22:02:51.705610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.705577    5580 generic.go:356] "Generic (PLEG): container finished" podID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerID="9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787" exitCode=0
Nov 02 22:02:51.705758 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.705679    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" event={"ID":"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5","Type":"ContainerDied","Data":"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"}
Nov 02 22:02:51.705758 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.705721    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt" event={"ID":"ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5","Type":"ContainerDied","Data":"322672d54a902b03d51dfa047afad623f31166dae2c5ba623c0f4d0958ecb7d5"}
Nov 02 22:02:51.705867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.705805    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"
Nov 02 22:02:51.707483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.707457    5580 generic.go:356] "Generic (PLEG): container finished" podID="617b5749-2132-4151-b919-aa96a10621ba" containerID="633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479" exitCode=0
Nov 02 22:02:51.707613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.707531    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"
Nov 02 22:02:51.707613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.707533    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" event={"ID":"617b5749-2132-4151-b919-aa96a10621ba","Type":"ContainerDied","Data":"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"}
Nov 02 22:02:51.707613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.707578    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7" event={"ID":"617b5749-2132-4151-b919-aa96a10621ba","Type":"ContainerDied","Data":"957609e9bc8a69c72bfffe9e2ace80eb89921fbb9d6fc7c4d6f5edfef29ac4a4"}
Nov 02 22:02:51.709299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.709280    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"
Nov 02 22:02:51.709299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.709283    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q" event={"ID":"d1fb38fa-c786-4e2c-b929-b03690d8a717","Type":"ContainerDied","Data":"ae0a8a61d8aac224da5ca878a5edf3d9ba705bf15465981ed1dff4569207536c"}
Nov 02 22:02:51.711257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.711233    5580 generic.go:356] "Generic (PLEG): container finished" podID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerID="ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110" exitCode=0
Nov 02 22:02:51.711357 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.711291    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"
Nov 02 22:02:51.711357 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.711310    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerDied","Data":"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"}
Nov 02 22:02:51.711357 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.711350    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4" event={"ID":"85d26d6f-037d-4880-ad82-5ac26e61ff20","Type":"ContainerDied","Data":"a1c86f0ac23c3f14eaff94da2216877cdbe50243cb826c8b1be511cd6571d6fe"}
Nov 02 22:02:51.713089 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.713068    5580 generic.go:356] "Generic (PLEG): container finished" podID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerID="051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193" exitCode=0
Nov 02 22:02:51.713175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.713109    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" event={"ID":"a574a4f3-284a-49cd-ae37-27313ca6c421","Type":"ContainerDied","Data":"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"}
Nov 02 22:02:51.713175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.713117    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"
Nov 02 22:02:51.713175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.713142    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj" event={"ID":"a574a4f3-284a-49cd-ae37-27313ca6c421","Type":"ContainerDied","Data":"dbcc52809d37b3de493c405900838065b90110aeec7eb7844df9ab27cd0a9ebb"}
Nov 02 22:02:51.714478 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.714454    5580 scope.go:122] "RemoveContainer" containerID="69fa266b24e998b8e284b6d95bc65cf1edaf40e34e2f7be59898804ad9f906fc"
Nov 02 22:02:51.724825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.724802    5580 scope.go:122] "RemoveContainer" containerID="9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"
Nov 02 22:02:51.735510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.735487    5580 scope.go:122] "RemoveContainer" containerID="9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"
Nov 02 22:02:51.735852 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:51.735830    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787\": container with ID starting with 9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787 not found: ID does not exist" containerID="9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"
Nov 02 22:02:51.735896 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.735866    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787"} err="failed to get container status \"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787\": rpc error: code = NotFound desc = could not find container \"9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787\": container with ID starting with 9a99baec50995a78012fe4fcee1746dd8b2bf0bd4341d5e421181d5a9d8d1787 not found: ID does not exist"
Nov 02 22:02:51.735896 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.735885    5580 scope.go:122] "RemoveContainer" containerID="633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"
Nov 02 22:02:51.754304 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.754273    5580 scope.go:122] "RemoveContainer" containerID="633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"
Nov 02 22:02:51.754774 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:51.754743    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479\": container with ID starting with 633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479 not found: ID does not exist" containerID="633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"
Nov 02 22:02:51.754882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.754785    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479"} err="failed to get container status \"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479\": rpc error: code = NotFound desc = could not find container \"633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479\": container with ID starting with 633fdbf8243280795229b2edc192d6362804cdc3aab740518b1fe88768758479 not found: ID does not exist"
Nov 02 22:02:51.754882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.754811    5580 scope.go:122] "RemoveContainer" containerID="551deb0c4c65134f5e7a298f43af60f480f874f737bf9c689959fd2e1768eef0"
Nov 02 22:02:51.766513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.766488    5580 scope.go:122] "RemoveContainer" containerID="ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"
Nov 02 22:02:51.777146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.777120    5580 scope.go:122] "RemoveContainer" containerID="4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"
Nov 02 22:02:51.789761 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.789728    5580 scope.go:122] "RemoveContainer" containerID="ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"
Nov 02 22:02:51.790206 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:51.790165    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110\": container with ID starting with ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110 not found: ID does not exist" containerID="ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"
Nov 02 22:02:51.790312 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.790218    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110"} err="failed to get container status \"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110\": rpc error: code = NotFound desc = could not find container \"ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110\": container with ID starting with ca1caa3b57ed5895f210c726718cdbfa5e613026080441c3fbf0ee2fa22f5110 not found: ID does not exist"
Nov 02 22:02:51.790312 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.790249    5580 scope.go:122] "RemoveContainer" containerID="4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"
Nov 02 22:02:51.790635 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:51.790607    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc\": container with ID starting with 4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc not found: ID does not exist" containerID="4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"
Nov 02 22:02:51.790741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.790644    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc"} err="failed to get container status \"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc\": rpc error: code = NotFound desc = could not find container \"4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc\": container with ID starting with 4dad3dabe97d6a7dad16dc6b6b7e1cae08bab7d0a4ba89be0286c602ada5a8fc not found: ID does not exist"
Nov 02 22:02:51.790741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.790669    5580 scope.go:122] "RemoveContainer" containerID="051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"
Nov 02 22:02:51.803776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.803748    5580 scope.go:122] "RemoveContainer" containerID="051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"
Nov 02 22:02:51.804234 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:51.804206    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193\": container with ID starting with 051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193 not found: ID does not exist" containerID="051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"
Nov 02 22:02:51.804336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.804243    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193"} err="failed to get container status \"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193\": rpc error: code = NotFound desc = could not find container \"051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193\": container with ID starting with 051307db0b6da83097c1341a91583b7b4bc1e5fde46009de0638394d43016193 not found: ID does not exist"
Nov 02 22:02:51.978278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.978238    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 22:02:51.982744 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:51.982712    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-wl9vf"]
Nov 02 22:02:52.093701 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.093655    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 22:02:52.099541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.099505    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-ztpj4"]
Nov 02 22:02:52.219491 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.219454    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 22:02:52.225093 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.225060    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-wwxtt"]
Nov 02 22:02:52.319277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.319171    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 22:02:52.322864 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.322832    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-hnppj"]
Nov 02 22:02:52.430946 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.430911    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 22:02:52.435140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.435108    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-c969q"]
Nov 02 22:02:52.547614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.547576    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 22:02:52.551519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.551483    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f48c4d-openshift-default-696bdfbcc5-vc6t7"]
Nov 02 22:02:52.670219 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.670180    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 22:02:52.677537 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.677493    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fb8112-openshift-default-6b6fcd5bf7-t9rxg"]
Nov 02 22:02:52.732314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.732280    5580 generic.go:356] "Generic (PLEG): container finished" podID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerID="325e5e8bdbb5285b2e00ce0cd849d7f32629eda6d2880f44c1a334aa46e4e82f" exitCode=0
Nov 02 22:02:52.732549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.732334    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerDied","Data":"325e5e8bdbb5285b2e00ce0cd849d7f32629eda6d2880f44c1a334aa46e4e82f"}
Nov 02 22:02:52.732549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.732389    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg" event={"ID":"16f720ff-0515-4447-a170-ddf3d263d6cf","Type":"ContainerDied","Data":"3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922"}
Nov 02 22:02:52.732549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.732414    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bc993d4b246ac2db71bb373422d5f22a530328d6f6343f794df782554989922"
Nov 02 22:02:52.732549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.732431    5580 scope.go:122] "RemoveContainer" containerID="3abfec66a24db755329be418a0294a3b1c5b9e8170e63afbd682571ec1579b6f"
Nov 02 22:02:52.734865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.734841    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 22:02:52.742663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.742068    5580 generic.go:356] "Generic (PLEG): container finished" podID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerID="2646d78b93c375cfc198f8ef84cd58b573d8147c042baeb46b2591d62a6f6ef0" exitCode=0
Nov 02 22:02:52.742663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.742172    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" event={"ID":"65135e0b-26b5-4bd9-80ff-d30d0f984713","Type":"ContainerDied","Data":"2646d78b93c375cfc198f8ef84cd58b573d8147c042baeb46b2591d62a6f6ef0"}
Nov 02 22:02:52.773986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.759369    5580 generic.go:356] "Generic (PLEG): container finished" podID="03f98fd7-f614-4f21-be06-d68d441ca799" containerID="45ce881187d21992e88f9f5da8e9c8b81177906e63d50b484e7a1ffcca01da6c" exitCode=0
Nov 02 22:02:52.773986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.759636    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" event={"ID":"03f98fd7-f614-4f21-be06-d68d441ca799","Type":"ContainerDied","Data":"45ce881187d21992e88f9f5da8e9c8b81177906e63d50b484e7a1ffcca01da6c"}
Nov 02 22:02:52.773986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.767800    5580 generic.go:356] "Generic (PLEG): container finished" podID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerID="3e2938c05fdc1fac8f15d43d72c271067d420af7e76bbacdd1d6cd29ba8b000c" exitCode=0
Nov 02 22:02:52.773986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.768004    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" event={"ID":"5d30f30a-d820-4f75-ab68-7a70d08658fa","Type":"ContainerDied","Data":"3e2938c05fdc1fac8f15d43d72c271067d420af7e76bbacdd1d6cd29ba8b000c"}
Nov 02 22:02:52.803222 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.803192    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 22:02:52.807996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.807255    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 22:02:52.815663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.815640    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 22:02:52.816179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816159    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816201    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816334    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816438    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.816517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816502    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816540    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816546    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.816576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816563    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816680    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816690    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.816728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816714    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.816883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816800    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.816883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.816864    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\") pod \"16f720ff-0515-4447-a170-ddf3d263d6cf\" (UID: \"16f720ff-0515-4447-a170-ddf3d263d6cf\") "
Nov 02 22:02:52.817630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.817601    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.817730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.817640    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.817730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.817657    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.817730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.817670    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.817730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.817604    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:52.823750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.823687    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "kube-api-access-sc7lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.823938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.823892    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:52.825335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.825308    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.830053 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.830022    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token" pod "16f720ff-0515-4447-a170-ddf3d263d6cf" (UID: "16f720ff-0515-4447-a170-ddf3d263d6cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.918361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918275    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.918361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918335    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.918543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918364    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.918543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918396    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.918543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918437    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.918543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918501    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918560    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918587    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918630    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918630    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918671    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.918756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918690    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918762    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918807    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918850    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918852    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918916    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.918980    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919031    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919071    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919087    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919102    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919133    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919171    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919175    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919206    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919251    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919297    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919331    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919377    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\") pod \"65135e0b-26b5-4bd9-80ff-d30d0f984713\" (UID: \"65135e0b-26b5-4bd9-80ff-d30d0f984713\") "
Nov 02 22:02:52.919485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919440    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919513    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\") pod \"5d30f30a-d820-4f75-ab68-7a70d08658fa\" (UID: \"5d30f30a-d820-4f75-ab68-7a70d08658fa\") "
Nov 02 22:02:52.920031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919580    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\") pod \"03f98fd7-f614-4f21-be06-d68d441ca799\" (UID: \"03f98fd7-f614-4f21-be06-d68d441ca799\") "
Nov 02 22:02:52.920031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919737    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.919743    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920062    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:52.920230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920065    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920324    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920473    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920536    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920619    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920642    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920657    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16f720ff-0515-4447-a170-ddf3d263d6cf-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920672    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920687    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920701    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f98fd7-f614-4f21-be06-d68d441ca799-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920716    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65135e0b-26b5-4bd9-80ff-d30d0f984713-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920730    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920744    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc7lb\" (UniqueName: \"kubernetes.io/projected/16f720ff-0515-4447-a170-ddf3d263d6cf-kube-api-access-sc7lb\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920764    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920778    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920794    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920808    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920809    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920822    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920837    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16f720ff-0515-4447-a170-ddf3d263d6cf-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.920914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.920856    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:52.922011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.921158    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:02:52.922011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.921262    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.923188 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923166    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:52.923328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923193    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "kube-api-access-fds72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.923423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923236    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:52.923423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923247    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.923532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923330    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:52.923532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923329    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "kube-api-access-fwz9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.923532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923371    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:02:52.923649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.923632    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "kube-api-access-qbf7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.924121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.924100    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token" pod "65135e0b-26b5-4bd9-80ff-d30d0f984713" (UID: "65135e0b-26b5-4bd9-80ff-d30d0f984713"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.924187 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.924118    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token" pod "03f98fd7-f614-4f21-be06-d68d441ca799" (UID: "03f98fd7-f614-4f21-be06-d68d441ca799"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:02:52.924571 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:52.924557    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy" pod "5d30f30a-d820-4f75-ab68-7a70d08658fa" (UID: "5d30f30a-d820-4f75-ab68-7a70d08658fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:02:53.022739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022679    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwz9f\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-kube-api-access-fwz9f\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.022739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022730    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.022739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022742    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.022739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022751    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022760    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022769    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022776    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f98fd7-f614-4f21-be06-d68d441ca799-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022784    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fds72\" (UniqueName: \"kubernetes.io/projected/5d30f30a-d820-4f75-ab68-7a70d08658fa-kube-api-access-fds72\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022793    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d30f30a-d820-4f75-ab68-7a70d08658fa-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022800    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbf7p\" (UniqueName: \"kubernetes.io/projected/03f98fd7-f614-4f21-be06-d68d441ca799-kube-api-access-qbf7p\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022808    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022817    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022824    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65135e0b-26b5-4bd9-80ff-d30d0f984713-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022834    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022843    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d30f30a-d820-4f75-ab68-7a70d08658fa-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.023034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.022851    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f98fd7-f614-4f21-be06-d68d441ca799-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:02:53.597722 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.597675    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" path="/var/lib/kubelet/pods/2e1ace86-270d-4edc-a433-689ce394e8d0/volumes"
Nov 02 22:02:53.598397 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.598375    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="617b5749-2132-4151-b919-aa96a10621ba" path="/var/lib/kubelet/pods/617b5749-2132-4151-b919-aa96a10621ba/volumes"
Nov 02 22:02:53.599089 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.599065    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" path="/var/lib/kubelet/pods/81fd7f81-3fc6-4fe6-8d1c-77a7e842313c/volumes"
Nov 02 22:02:53.599852 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.599827    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" path="/var/lib/kubelet/pods/85d26d6f-037d-4880-ad82-5ac26e61ff20/volumes"
Nov 02 22:02:53.600351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.600330    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" path="/var/lib/kubelet/pods/a574a4f3-284a-49cd-ae37-27313ca6c421/volumes"
Nov 02 22:02:53.600864 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.600844    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" path="/var/lib/kubelet/pods/d1fb38fa-c786-4e2c-b929-b03690d8a717/volumes"
Nov 02 22:02:53.601639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.601618    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" path="/var/lib/kubelet/pods/ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5/volumes"
Nov 02 22:02:53.760004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.759933    5580 patch_prober.go:29] interesting pod/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:02:53.760233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.760104    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:02:53.781873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.781840    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"
Nov 02 22:02:53.781873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.781866    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl" event={"ID":"5d30f30a-d820-4f75-ab68-7a70d08658fa","Type":"ContainerDied","Data":"e885c379f7ceaac893a04aaf8b49bbc7db9f89381b7d46dbcb969945cab5e72f"}
Nov 02 22:02:53.782127 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.781926    5580 scope.go:122] "RemoveContainer" containerID="3e2938c05fdc1fac8f15d43d72c271067d420af7e76bbacdd1d6cd29ba8b000c"
Nov 02 22:02:53.783815 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.783791    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"
Nov 02 22:02:53.785994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.785939    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"
Nov 02 22:02:53.786123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.786004    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs" event={"ID":"65135e0b-26b5-4bd9-80ff-d30d0f984713","Type":"ContainerDied","Data":"e1c45b0835345be957c0b98686a61b38e8fc214be69e7df3d1c63610229d5aa9"}
Nov 02 22:02:53.787949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.787919    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4" event={"ID":"03f98fd7-f614-4f21-be06-d68d441ca799","Type":"ContainerDied","Data":"f15b99428d8647f7d3510bf39c04b0e00c71b2c2322f85833b59256626f18114"}
Nov 02 22:02:53.788063 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.788005    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"
Nov 02 22:02:53.794575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.794550    5580 scope.go:122] "RemoveContainer" containerID="2646d78b93c375cfc198f8ef84cd58b573d8147c042baeb46b2591d62a6f6ef0"
Nov 02 22:02:53.807730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:53.807706    5580 scope.go:122] "RemoveContainer" containerID="45ce881187d21992e88f9f5da8e9c8b81177906e63d50b484e7a1ffcca01da6c"
Nov 02 22:02:54.007697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.007653    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 22:02:54.010925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.010853    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-9lpxg"]
Nov 02 22:02:54.118609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.118568    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 22:02:54.121716 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.121682    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-vhvhs"]
Nov 02 22:02:54.228552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.228516    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 22:02:54.234244 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.234211    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-fscx4"]
Nov 02 22:02:54.341897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.341807    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 22:02:54.346071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:54.346040    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-96c052-openshift-default-86677f975c-qvbtl"]
Nov 02 22:02:55.594747 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:55.594708    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" path="/var/lib/kubelet/pods/03f98fd7-f614-4f21-be06-d68d441ca799/volumes"
Nov 02 22:02:55.595355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:55.595338    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" path="/var/lib/kubelet/pods/16f720ff-0515-4447-a170-ddf3d263d6cf/volumes"
Nov 02 22:02:55.595839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:55.595823    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" path="/var/lib/kubelet/pods/5d30f30a-d820-4f75-ab68-7a70d08658fa/volumes"
Nov 02 22:02:55.597341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:02:55.597308    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" path="/var/lib/kubelet/pods/65135e0b-26b5-4bd9-80ff-d30d0f984713/volumes"
Nov 02 22:02:56.311891 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:02:56.311858    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:03:02.000987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:02.000943    5580 scope.go:122] "RemoveContainer" containerID="325e5e8bdbb5285b2e00ce0cd849d7f32629eda6d2880f44c1a334aa46e4e82f"
Nov 02 22:03:12.742667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:12.742629    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:03:15.326181 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326142    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:03:15.326839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326822    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy"
Nov 02 22:03:15.326839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326840    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326859    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326875    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326887    5580 cpu_manager.go:414] "containerMap: removing container" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326893    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326904    5580 cpu_manager.go:414] "containerMap: removing container" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326909    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326920    5580 cpu_manager.go:414] "containerMap: removing container" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326925    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326939    5580 cpu_manager.go:414] "containerMap: removing container" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326943    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326966    5580 cpu_manager.go:414] "containerMap: removing container" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy"
Nov 02 22:03:15.326949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326971    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326982    5580 cpu_manager.go:414] "containerMap: removing container" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326987    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326994    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.326999    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327007    5580 cpu_manager.go:414] "containerMap: removing container" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327012    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327022    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327026    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327036    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327040    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327049    5580 cpu_manager.go:414] "containerMap: removing container" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327054    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327063    5580 cpu_manager.go:414] "containerMap: removing container" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327068    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327166    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="65135e0b-26b5-4bd9-80ff-d30d0f984713" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327175    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327184    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff5ca6d9-3fb1-4e4a-85eb-a4da3ba673f5" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327190    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e1ace86-270d-4edc-a433-689ce394e8d0" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327197    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d30f30a-d820-4f75-ab68-7a70d08658fa" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327203    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a574a4f3-284a-49cd-ae37-27313ca6c421" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327209    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327214    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="81fd7f81-3fc6-4fe6-8d1c-77a7e842313c" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327219    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1fb38fa-c786-4e2c-b929-b03690d8a717" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327225    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c68e7e0-c1f4-4df9-af1a-04c46b3be70f" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327231    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="617b5749-2132-4151-b919-aa96a10621ba" containerName="istio-proxy"
Nov 02 22:03:15.327551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.327238    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="03f98fd7-f614-4f21-be06-d68d441ca799" containerName="istio-proxy"
Nov 02 22:03:15.331000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.330947    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.367737 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.367702    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.368029 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.368011    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8dcs\" (UniqueName: \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.368101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.368048    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.368101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.368074    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.432363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.432330    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-75577e-serving-cert\""
Nov 02 22:03:15.435848 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.435821    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:03:15.469402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.469372    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8dcs\" (UniqueName: \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.469600 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.469418    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.469600 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.469448    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.469600 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.469485    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.469873 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:15.469853    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-serving-cert: secret "maas-api-e2e-shared-b-w5-75577e-serving-cert" not found
Nov 02 22:03:15.470346 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:15.469941    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls podName:be91195f-5b0e-4bcb-a7ea-1b57192335ce nodeName:}" failed. No retries permitted until 2026-11-02 22:03:15.969915966 +0000 UTC m=+1694.486411732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls") pod "maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" (UID: "be91195f-5b0e-4bcb-a7ea-1b57192335ce") : secret "maas-api-e2e-shared-b-w5-75577e-serving-cert" not found
Nov 02 22:03:15.470519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.470489    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.472209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.472183    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.528514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.528479    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8dcs\" (UniqueName: \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.979602 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.979566    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:15.982119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:15.982092    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:16.245715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:16.245618    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:16.626971 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:16.626924    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:03:16.628313 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:03:16.628267    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe91195f_5b0e_4bcb_a7ea_1b57192335ce.slice/crio-9f3bafbd0554ca49e77c33ea3a5fda6d3d892616ca6d131c3271fd09f3ad3a52 WatchSource:0}: Error finding container 9f3bafbd0554ca49e77c33ea3a5fda6d3d892616ca6d131c3271fd09f3ad3a52: Status 404 returned error can't find the container with id 9f3bafbd0554ca49e77c33ea3a5fda6d3d892616ca6d131c3271fd09f3ad3a52
Nov 02 22:03:16.629911 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:16.629892    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:03:16.956924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:16.956886    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" event={"ID":"be91195f-5b0e-4bcb-a7ea-1b57192335ce","Type":"ContainerStarted","Data":"9f3bafbd0554ca49e77c33ea3a5fda6d3d892616ca6d131c3271fd09f3ad3a52"}
Nov 02 22:03:17.965161 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:17.965120    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" event={"ID":"be91195f-5b0e-4bcb-a7ea-1b57192335ce","Type":"ContainerStarted","Data":"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"}
Nov 02 22:03:17.965628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:17.965189    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:18.081951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:18.081889    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" podStartSLOduration=2.185227093 podStartE2EDuration="3.081871751s" totalImagesPullingTime="896.644658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:03:18.078446012 +0000 UTC m=+1696.594941809" watchObservedRunningTime="2026-11-02 22:03:18.081871751 +0000 UTC m=+1696.598367572"
Nov 02 22:03:20.092850 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.092815    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:03:20.093690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.093670    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="16f720ff-0515-4447-a170-ddf3d263d6cf" containerName="istio-proxy"
Nov 02 22:03:20.093748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.093695    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="85d26d6f-037d-4880-ad82-5ac26e61ff20" containerName="istio-proxy"
Nov 02 22:03:20.097504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.097479    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.150563 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.150524    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqggx\" (UniqueName: \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.150777 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.150584    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.150777 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.150745    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.150989 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.150939    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.199095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.199041    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4fc235-serving-cert\""
Nov 02 22:03:20.202441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.202402    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:03:20.252334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.252297    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.252531 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:20.252449    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-serving-cert: secret "maas-api-e2e-api-a-4fc235-serving-cert" not found
Nov 02 22:03:20.252531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.252493    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqggx\" (UniqueName: \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.252531 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:20.252514    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls podName:ce2b6302-47b8-4974-8b87-71ef778ad3f0 nodeName:}" failed. No retries permitted until 2026-11-02 22:03:20.752497229 +0000 UTC m=+1699.268992996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls") pod "maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" (UID: "ce2b6302-47b8-4974-8b87-71ef778ad3f0") : secret "maas-api-e2e-api-a-4fc235-serving-cert" not found
Nov 02 22:03:20.252709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.252563    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.252709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.252612    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.253349 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.253327    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.255235 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.255210    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.309538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.309508    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqggx\" (UniqueName: \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.759733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.759696    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:20.762240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:20.762212    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") pod \"maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:21.011709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:21.011603    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:21.391904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:21.391873    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:03:21.392945 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:03:21.392920    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2b6302_47b8_4974_8b87_71ef778ad3f0.slice/crio-1ab473876f52c0759a977f1899aff844eed3c4d18f718872bbb1ca9f1af7dc83 WatchSource:0}: Error finding container 1ab473876f52c0759a977f1899aff844eed3c4d18f718872bbb1ca9f1af7dc83: Status 404 returned error can't find the container with id 1ab473876f52c0759a977f1899aff844eed3c4d18f718872bbb1ca9f1af7dc83
Nov 02 22:03:21.995124 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:21.995083    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" event={"ID":"ce2b6302-47b8-4974-8b87-71ef778ad3f0","Type":"ContainerStarted","Data":"1ab473876f52c0759a977f1899aff844eed3c4d18f718872bbb1ca9f1af7dc83"}
Nov 02 22:03:23.003565 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:23.003527    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" event={"ID":"ce2b6302-47b8-4974-8b87-71ef778ad3f0","Type":"ContainerStarted","Data":"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"}
Nov 02 22:03:23.004011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:23.003681    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:23.978417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:23.978386    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:03:24.090925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:24.090856    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" podStartSLOduration=3.129923824 podStartE2EDuration="4.09083764s" totalImagesPullingTime="960.913816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:03:23.117109733 +0000 UTC m=+1701.633605507" watchObservedRunningTime="2026-11-02 22:03:24.09083764 +0000 UTC m=+1702.607333414"
Nov 02 22:03:26.312972 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:26.312920    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:03:29.016436 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:29.016405    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:03:38.324980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.324181    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 22:03:38.360112 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.360063    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:03:38.493981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.493114    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:03:38.634822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.634724    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:03:38.649847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.649769    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.705351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.705320    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-267d89-openshift-default-dockercfg-czbd5\""
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916491    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916588    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7ksl\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916641    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916790    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916838    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916881    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916949    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.916987    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:38.924948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:38.917008    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.073983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.067306    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.074363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.074327    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.074531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.074512    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.074992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.074945    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.075172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.075154    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.075304 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.075287    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.075782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.075447    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.095014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.076040    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7ksl\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.102988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.099188    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.102988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.092529    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.102988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.092044    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.102988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.101904    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.122980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.110248    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:03:39.122980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.111950    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.122980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.118611    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.149026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.141441    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.172779 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.171557    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.175651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.175482    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7ksl\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.179596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.179557    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\") pod \"e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") " pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.292647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.292602    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:39.726614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.726584    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:03:39.727080 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:03:39.727050    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3762c23d_3be1_47c7_bfda_024fd935d42e.slice/crio-5e3a5ad10d3653e1bcdf72e08fc9840191a270f0edcd661df45c5204efd657db WatchSource:0}: Error finding container 5e3a5ad10d3653e1bcdf72e08fc9840191a270f0edcd661df45c5204efd657db: Status 404 returned error can't find the container with id 5e3a5ad10d3653e1bcdf72e08fc9840191a270f0edcd661df45c5204efd657db
Nov 02 22:03:39.729889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.729856    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:03:39.729982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.729928    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:03:39.730027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:39.730001    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:03:40.173744 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:40.173707    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerStarted","Data":"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"}
Nov 02 22:03:40.174234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:40.173766    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerStarted","Data":"5e3a5ad10d3653e1bcdf72e08fc9840191a270f0edcd661df45c5204efd657db"}
Nov 02 22:03:40.290052 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:40.289933    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podStartSLOduration=2.289913675 podStartE2EDuration="2.289913675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:03:40.289431342 +0000 UTC m=+1718.805927142" watchObservedRunningTime="2026-11-02 22:03:40.289913675 +0000 UTC m=+1718.806409448"
Nov 02 22:03:40.293247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:40.293226    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:40.299516 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:40.299485    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:41.180547 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:41.180513    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:41.181482 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:41.181466    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:03:49.111061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.111015    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:03:49.116541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.116511    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.171441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.169314    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-267d89\""
Nov 02 22:03:49.174058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.174005    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:03:49.218872 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.218840    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-267d89-dockercfg-pjtlv\""
Nov 02 22:03:49.281004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.280950    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.281250 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.281038    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9hss\" (UniqueName: \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.383158 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.383063    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.383158 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.383115    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9hss\" (UniqueName: \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.384038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.384010    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.441625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.441595    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9hss\" (UniqueName: \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\") pod \"payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:49.739120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:49.739077    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:50.121476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:50.121439    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:03:50.246137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:50.246100    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" event={"ID":"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf","Type":"ContainerStarted","Data":"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"}
Nov 02 22:03:50.246327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:50.246163    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" event={"ID":"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf","Type":"ContainerStarted","Data":"263a543b02d749697a034a79e90b024b3599fcb42a77eb8343a57590c3a22730"}
Nov 02 22:03:50.246327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:50.246201    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:03:50.359440 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:03:50.359378    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" podStartSLOduration=1.35935806 podStartE2EDuration="1.35935806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:03:50.357487533 +0000 UTC m=+1728.873983308" watchObservedRunningTime="2026-11-02 22:03:50.35935806 +0000 UTC m=+1728.875853845"
Nov 02 22:03:56.312176 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:03:56.312133    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:04:00.715760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:00.715720    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 22:04:00.716256 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:00.715934    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy" containerID="cri-o://07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8" gracePeriod=30
Nov 02 22:04:01.255563 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:01.255531    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:04:03.137780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.137740    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:04:03.143780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.143754    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213063 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213021    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213082    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnn27\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213270    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213302    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213383    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213423    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213543    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213615 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213604    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.213654 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.213644    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.228928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.228898    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-3eb190-openshift-default-dockercfg-njcrw\""
Nov 02 22:04:03.254141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.254102    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:04:03.315161 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315129    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315188    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnn27\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315283    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315306    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315345    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315571 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315375    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315571 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315480    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315679 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315604    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.315679 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.315652    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.316278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.316252    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.317261 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.317229    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.317699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.317675    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.318109 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.318085    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.319312 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.318749    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.321595 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.321544    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.321754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.321737    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.373334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.373291    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnn27\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.373807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.373787    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\") pod \"e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") " pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.460345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.460304    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:03.931135 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:03.931106    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04e413b_81eb_4b44_bbba_a9fa2b17259c.slice/crio-4469cf703895ee8b26cc34d00ec4ce289e326e9a3f0e2b09e3da1761bd40ba9a WatchSource:0}: Error finding container 4469cf703895ee8b26cc34d00ec4ce289e326e9a3f0e2b09e3da1761bd40ba9a: Status 404 returned error can't find the container with id 4469cf703895ee8b26cc34d00ec4ce289e326e9a3f0e2b09e3da1761bd40ba9a
Nov 02 22:04:03.932073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.932043    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:04:03.935245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.935205    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:03.935365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.935303    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:03.935365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:03.935348    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:04.366711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:04.366599    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerStarted","Data":"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"}
Nov 02 22:04:04.366711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:04.366661    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerStarted","Data":"4469cf703895ee8b26cc34d00ec4ce289e326e9a3f0e2b09e3da1761bd40ba9a"}
Nov 02 22:04:04.460839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:04.460800    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:04.469114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:04.469078    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:04.483433 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:04.483381    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podStartSLOduration=1.483365813 podStartE2EDuration="1.483365813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:04.481614356 +0000 UTC m=+1742.998110164" watchObservedRunningTime="2026-11-02 22:04:04.483365813 +0000 UTC m=+1742.999861585"
Nov 02 22:04:05.373254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:05.373219    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:05.382510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:05.378940    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:04:06.002996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.002933    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 22:04:06.162496 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162455    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162570    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162616    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162664    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162732    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162858    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.162972 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162917    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:06.162972 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.162938    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.163093 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163012    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.163149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163135    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\") pod \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\" (UID: \"0cf32480-e907-4c7c-a0db-bbb3cf49e8be\") "
Nov 02 22:04:06.163207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163191    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:06.163293 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163260    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:04:06.163533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163509    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:06.163728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.163701    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:06.164178 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.164152    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.164280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.164184    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.164280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.164202    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.164280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.164218    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.164280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.164233    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.166497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.166472    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:06.166603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.166512    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "kube-api-access-9kwbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:06.166713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.166683    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:04:06.170360 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.170328    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy" pod "0cf32480-e907-4c7c-a0db-bbb3cf49e8be" (UID: "0cf32480-e907-4c7c-a0db-bbb3cf49e8be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:06.265847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.265647    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.265847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.265681    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.265847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.265694    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.265847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.265706    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kwbd\" (UniqueName: \"kubernetes.io/projected/0cf32480-e907-4c7c-a0db-bbb3cf49e8be-kube-api-access-9kwbd\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:06.381662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.381625    5580 generic.go:356] "Generic (PLEG): container finished" podID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerID="07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8" exitCode=0
Nov 02 22:04:06.382150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.381681    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"
Nov 02 22:04:06.382150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.381704    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" event={"ID":"0cf32480-e907-4c7c-a0db-bbb3cf49e8be","Type":"ContainerDied","Data":"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"}
Nov 02 22:04:06.382150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.381768    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq" event={"ID":"0cf32480-e907-4c7c-a0db-bbb3cf49e8be","Type":"ContainerDied","Data":"b9c01441790b35038694f8b71c85f555c5bfc5522f17c117f8f819b33f95d666"}
Nov 02 22:04:06.382150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.381800    5580 scope.go:122] "RemoveContainer" containerID="07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"
Nov 02 22:04:06.393463 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.393439    5580 scope.go:122] "RemoveContainer" containerID="07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"
Nov 02 22:04:06.397770 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:04:06.397729    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8\": container with ID starting with 07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8 not found: ID does not exist" containerID="07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"
Nov 02 22:04:06.397929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.397778    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8"} err="failed to get container status \"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8\": rpc error: code = NotFound desc = could not find container \"07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8\": container with ID starting with 07bf6aca42b6e81682d1405b032c1100c5b52e0d941d17596e8a8586cf6327b8 not found: ID does not exist"
Nov 02 22:04:06.544938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.544859    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 22:04:06.550375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:06.550343    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-td5fq"]
Nov 02 22:04:07.595360 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:07.595320    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" path="/var/lib/kubelet/pods/0cf32480-e907-4c7c-a0db-bbb3cf49e8be/volumes"
Nov 02 22:04:11.027036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.026989    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:04:11.028108 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.028082    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy"
Nov 02 22:04:11.028108 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.028109    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy"
Nov 02 22:04:11.028346 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.028332    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cf32480-e907-4c7c-a0db-bbb3cf49e8be" containerName="istio-proxy"
Nov 02 22:04:11.034216 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.034192    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.069071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.069032    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:04:11.116177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.111463    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-3eb190\""
Nov 02 22:04:11.116177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.111616    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-3eb190-dockercfg-r5bdc\""
Nov 02 22:04:11.136236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.136199    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.136455 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.136422    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tww6z\" (UniqueName: \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.238395 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.238354    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tww6z\" (UniqueName: \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.238605 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.238587    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.239342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.239320    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.294813 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.294730    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tww6z\" (UniqueName: \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\") pod \"payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.354184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.354142    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:11.749419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:11.749383    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:04:11.750334 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:11.750298    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebdb250f_d4bb_46cf_b337_266445d6251e.slice/crio-d7f5ddf4d7b1da7f8359d0fcdc1ad3dba69c03ce70f7eadec40c1bf5c66e4e99 WatchSource:0}: Error finding container d7f5ddf4d7b1da7f8359d0fcdc1ad3dba69c03ce70f7eadec40c1bf5c66e4e99: Status 404 returned error can't find the container with id d7f5ddf4d7b1da7f8359d0fcdc1ad3dba69c03ce70f7eadec40c1bf5c66e4e99
Nov 02 22:04:12.430335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:12.430296    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" event={"ID":"ebdb250f-d4bb-46cf-b337-266445d6251e","Type":"ContainerStarted","Data":"c4808bbdcd05b9851660323a85855ff2d4b7feb333a6fa3a07a43fd4cd90a6a3"}
Nov 02 22:04:12.430786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:12.430370    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:12.430786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:12.430387    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" event={"ID":"ebdb250f-d4bb-46cf-b337-266445d6251e","Type":"ContainerStarted","Data":"d7f5ddf4d7b1da7f8359d0fcdc1ad3dba69c03ce70f7eadec40c1bf5c66e4e99"}
Nov 02 22:04:12.546413 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:12.546352    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" podStartSLOduration=2.546336671 podStartE2EDuration="2.546336671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:12.544412361 +0000 UTC m=+1751.060908129" watchObservedRunningTime="2026-11-02 22:04:12.546336671 +0000 UTC m=+1751.062832461"
Nov 02 22:04:15.708359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:15.708324    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 22:04:15.708776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:15.708548    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy" containerID="cri-o://6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b" gracePeriod=30
Nov 02 22:04:20.999642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:20.999613    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 22:04:21.107418 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107382    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107450    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107490    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107569    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107643    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107694    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.107797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107748    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.108024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107860    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.108024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107972    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\") pod \"0d692a6c-6165-4ab6-b343-edd544e293d0\" (UID: \"0d692a6c-6165-4ab6-b343-edd544e293d0\") "
Nov 02 22:04:21.108024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107947    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:04:21.108024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.107936    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:21.108371 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108022    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:21.108371 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108196    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:21.108477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108302    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:21.108874 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108852    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.108949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108880    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.108949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108895    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.108949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108910    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.108949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.108923    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d692a6c-6165-4ab6-b343-edd544e293d0-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.109895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.109873    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:04:21.110234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.110214    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "kube-api-access-w4xsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:21.110327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.110299    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:04:21.110385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.110334    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy" pod "0d692a6c-6165-4ab6-b343-edd544e293d0" (UID: "0d692a6c-6165-4ab6-b343-edd544e293d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:04:21.210832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.210794    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.210832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.210828    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.210832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.210837    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d692a6c-6165-4ab6-b343-edd544e293d0-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.211094 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.210847    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4xsf\" (UniqueName: \"kubernetes.io/projected/0d692a6c-6165-4ab6-b343-edd544e293d0-kube-api-access-w4xsf\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:04:21.493609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.493570    5580 generic.go:356] "Generic (PLEG): container finished" podID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerID="6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b" exitCode=0
Nov 02 22:04:21.493808 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.493630    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"
Nov 02 22:04:21.493808 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.493650    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" event={"ID":"0d692a6c-6165-4ab6-b343-edd544e293d0","Type":"ContainerDied","Data":"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"}
Nov 02 22:04:21.493808 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.493710    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd" event={"ID":"0d692a6c-6165-4ab6-b343-edd544e293d0","Type":"ContainerDied","Data":"7b8c835a2df90d7bfb6d903679aeeba93a6bd3e3237136877c2ef8d451d16488"}
Nov 02 22:04:21.493808 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.493734    5580 scope.go:122] "RemoveContainer" containerID="6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"
Nov 02 22:04:21.509140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.508984    5580 scope.go:122] "RemoveContainer" containerID="6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"
Nov 02 22:04:21.509393 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:04:21.509372    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b\": container with ID starting with 6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b not found: ID does not exist" containerID="6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"
Nov 02 22:04:21.509483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.509404    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b"} err="failed to get container status \"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b\": rpc error: code = NotFound desc = could not find container \"6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b\": container with ID starting with 6bf74d3a4054f42dff350e475dcf0bd05c9b325d694b1254341ba77f2b0d886b not found: ID does not exist"
Nov 02 22:04:21.658422 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.658369    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 22:04:21.708046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:21.708006    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-4t4nd"]
Nov 02 22:04:22.530872 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:22.530832    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:04:23.440376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:23.440346    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:04:23.595463 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:23.595432    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" path="/var/lib/kubelet/pods/0d692a6c-6165-4ab6-b343-edd544e293d0/volumes"
Nov 02 22:04:26.313251 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:04:26.313208    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:04:36.685276 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.685240    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:04:36.685896 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.685883    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy"
Nov 02 22:04:36.685939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.685898    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy"
Nov 02 22:04:36.686075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.686062    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d692a6c-6165-4ab6-b343-edd544e293d0" containerName="istio-proxy"
Nov 02 22:04:36.690613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.690594    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.814102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.806365    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-224f53-openshift-default-dockercfg-7kbx9\""
Nov 02 22:04:36.829650 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.829596    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863641    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863704    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863774    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863828    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863883    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.863920    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.864065    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.864121    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.869980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.864216    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965332 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965218    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965332 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965286    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965341    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965410    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965432    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965472    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965509    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965579 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965558    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.965881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.965592    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.966066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.966037    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.966401 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.966290    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.966519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.966332    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.966519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.966349    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.966732 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.966710    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.968840 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.968804    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:36.969091 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:36.968971    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:37.021825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.021793    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:37.022005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.021933    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\") pod \"e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") " pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:37.306155 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.306059    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:37.751747 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.751709    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:04:37.754810 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:37.754773    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd26ff062_e4ec_4516_9691_799cf88e4626.slice/crio-73680907fa4b42fcf1f249fe478e1f8059e6ba37c473a39828cd7183dece89cc WatchSource:0}: Error finding container 73680907fa4b42fcf1f249fe478e1f8059e6ba37c473a39828cd7183dece89cc: Status 404 returned error can't find the container with id 73680907fa4b42fcf1f249fe478e1f8059e6ba37c473a39828cd7183dece89cc
Nov 02 22:04:37.758465 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.758346    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:37.758629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.758484    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:37.758629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:37.758563    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:38.622832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:38.622796    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerStarted","Data":"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"}
Nov 02 22:04:38.623050 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:38.622860    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerStarted","Data":"73680907fa4b42fcf1f249fe478e1f8059e6ba37c473a39828cd7183dece89cc"}
Nov 02 22:04:38.838046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:38.837935    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podStartSLOduration=2.837917133 podStartE2EDuration="2.837917133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:38.786737057 +0000 UTC m=+1777.303232814" watchObservedRunningTime="2026-11-02 22:04:38.837917133 +0000 UTC m=+1777.354412906"
Nov 02 22:04:39.306309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:39.306263    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:39.313281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:39.313251    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:39.629718 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:39.629629    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:39.630685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:39.630663    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:04:48.280732 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.280695    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:04:48.287446 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.287413    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.336946 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.336914    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:04:48.352412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.352354    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.352631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.352574    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwmqc\" (UniqueName: \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.382685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.382651    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-224f53\""
Nov 02 22:04:48.383392 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.383369    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-224f53-dockercfg-5tnm4\""
Nov 02 22:04:48.457979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.456048    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwmqc\" (UniqueName: \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.457979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.456440    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.457979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.457453    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.538217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.536565    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwmqc\" (UniqueName: \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\") pod \"payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.603801 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.603758    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:48.993208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:48.993181    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:04:48.993386 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:48.993273    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639de4ba_63b5_45a3_a72e_f88a676ffa5d.slice/crio-bf99e10c75daf685d5f1eab4def8d40b6ca79da817cce347537b270251b5e23f WatchSource:0}: Error finding container bf99e10c75daf685d5f1eab4def8d40b6ca79da817cce347537b270251b5e23f: Status 404 returned error can't find the container with id bf99e10c75daf685d5f1eab4def8d40b6ca79da817cce347537b270251b5e23f
Nov 02 22:04:49.144240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.144204    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:04:49.148593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.148571    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.220503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.220452    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-587629-openshift-default-dockercfg-65vk6\""
Nov 02 22:04:49.267044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.266512    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381078    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381180    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381233    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381268    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381370    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381436    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381515    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381612    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsm62\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.381976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.381645    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.485630    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.485725    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.485753    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.485777    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487059    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487062    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487166    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487253    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.487466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487352    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsm62\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.488244 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487446    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.488381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.487379    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.491499 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.491455    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.491736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.491592    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.491996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.491952    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.492248 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.492224    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.497767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.497727    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.549406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.549289    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.550209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.550185    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsm62\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\") pod \"e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") " pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.685305 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.685264    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:49.724995 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.724903    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" event={"ID":"639de4ba-63b5-45a3-a72e-f88a676ffa5d","Type":"ContainerStarted","Data":"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"}
Nov 02 22:04:49.731174 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.731123    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" event={"ID":"639de4ba-63b5-45a3-a72e-f88a676ffa5d","Type":"ContainerStarted","Data":"bf99e10c75daf685d5f1eab4def8d40b6ca79da817cce347537b270251b5e23f"}
Nov 02 22:04:49.732111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.732088    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:04:49.848802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:49.848671    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" podStartSLOduration=1.848649644 podStartE2EDuration="1.848649644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:49.84441651 +0000 UTC m=+1788.360912283" watchObservedRunningTime="2026-11-02 22:04:49.848649644 +0000 UTC m=+1788.365145418"
Nov 02 22:04:50.171085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.171051    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:04:50.171604 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:50.171579    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4389b173_7d78_48c9_926f_065f5495ac7c.slice/crio-865e88847761d03561bed766aa79eea0ae25012f01ed37f55a921560cab26350 WatchSource:0}: Error finding container 865e88847761d03561bed766aa79eea0ae25012f01ed37f55a921560cab26350: Status 404 returned error can't find the container with id 865e88847761d03561bed766aa79eea0ae25012f01ed37f55a921560cab26350
Nov 02 22:04:50.180074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.180032    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:50.180212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.180125    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:50.180212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.180168    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:04:50.734061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.734023    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerStarted","Data":"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"}
Nov 02 22:04:50.734577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.734080    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerStarted","Data":"865e88847761d03561bed766aa79eea0ae25012f01ed37f55a921560cab26350"}
Nov 02 22:04:50.853490 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:50.853434    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podStartSLOduration=1.853417387 podStartE2EDuration="1.853417387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:50.851022492 +0000 UTC m=+1789.367518265" watchObservedRunningTime="2026-11-02 22:04:50.853417387 +0000 UTC m=+1789.369913160"
Nov 02 22:04:51.664659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:51.664623    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 22:04:51.685638 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:51.685600    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:51.692816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:51.692783    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:51.742101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:51.742070    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:51.743324 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:51.743303    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:04:56.313385 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:04:56.313344    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:04:56.665912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:56.665801    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing" containerID="cri-o://8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef" gracePeriod=25
Nov 02 22:04:57.900212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:57.900168    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:04:58.375909 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.375876    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:04:58.381320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.381256    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.438846 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.438804    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:04:58.488996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.478162    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d6z8\" (UniqueName: \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.488996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.478261    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.488996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.483758    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-587629-dockercfg-rn897\""
Nov 02 22:04:58.488996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.483843    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-587629\""
Nov 02 22:04:58.580982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.580264    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d6z8\" (UniqueName: \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.580982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.580357    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.606948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.581538    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.680508 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.676292    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d6z8\" (UniqueName: \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\") pod \"payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:58.706675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:58.706594    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:59.101476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:59.101439    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:04:59.102040 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:04:59.102014    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0732474_c0ee_41c3_9e31_a79383d1d776.slice/crio-cc7ce56c9fd3fe7ac807996173fc4dd6912623ba5616d99705eb29a2ce5c7888 WatchSource:0}: Error finding container cc7ce56c9fd3fe7ac807996173fc4dd6912623ba5616d99705eb29a2ce5c7888: Status 404 returned error can't find the container with id cc7ce56c9fd3fe7ac807996173fc4dd6912623ba5616d99705eb29a2ce5c7888
Nov 02 22:04:59.806801 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:59.806759    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" event={"ID":"b0732474-c0ee-41c3-9e31-a79383d1d776","Type":"ContainerStarted","Data":"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"}
Nov 02 22:04:59.807035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:59.806826    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" event={"ID":"b0732474-c0ee-41c3-9e31-a79383d1d776","Type":"ContainerStarted","Data":"cc7ce56c9fd3fe7ac807996173fc4dd6912623ba5616d99705eb29a2ce5c7888"}
Nov 02 22:04:59.807035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:59.806856    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:04:59.921906 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:04:59.921847    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" podStartSLOduration=1.921831458 podStartE2EDuration="1.921831458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:59.919123402 +0000 UTC m=+1798.435619204" watchObservedRunningTime="2026-11-02 22:04:59.921831458 +0000 UTC m=+1798.438327230"
Nov 02 22:05:00.737768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:00.737741    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:05:02.084026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:02.083996    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:05:02.087059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:02.087036    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:05:02.094527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:02.094501    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:05:02.098887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:02.098850    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:05:07.900536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:07.900489    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:05:10.819555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:10.819505    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:05:12.122047 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:12.112065    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:12.122047 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:12.112615    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:16.948720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:16.948688    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 22:05:17.029059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.029021    5580 generic.go:356] "Generic (PLEG): container finished" podID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerID="8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef" exitCode=0
Nov 02 22:05:17.029239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.029084    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"
Nov 02 22:05:17.029239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.029101    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" event={"ID":"82a31b1e-ae7e-41f9-b9b0-22ab255a729b","Type":"ContainerDied","Data":"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"}
Nov 02 22:05:17.029239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.029158    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7" event={"ID":"82a31b1e-ae7e-41f9-b9b0-22ab255a729b","Type":"ContainerDied","Data":"44f445f424436523cee685246d3235076d84f12dafaaaa389e4d77d734656853"}
Nov 02 22:05:17.029239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.029180    5580 scope.go:122] "RemoveContainer" containerID="8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"
Nov 02 22:05:17.039452 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.039428    5580 scope.go:122] "RemoveContainer" containerID="8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"
Nov 02 22:05:17.039810 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:17.039785    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef\": container with ID starting with 8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef not found: ID does not exist" containerID="8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"
Nov 02 22:05:17.039889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.039825    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef"} err="failed to get container status \"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef\": rpc error: code = NotFound desc = could not find container \"8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef\": container with ID starting with 8b3a4dc037f73a4690eaded5ceb608a1f34f6dedaddb755327569cd579b77cef not found: ID does not exist"
Nov 02 22:05:17.060432 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.060396    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\" (UniqueName: \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\") pod \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") "
Nov 02 22:05:17.060606 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.060500    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\") pod \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\" (UID: \"82a31b1e-ae7e-41f9-b9b0-22ab255a729b\") "
Nov 02 22:05:17.060840 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.060818    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume" pod "82a31b1e-ae7e-41f9-b9b0-22ab255a729b" (UID: "82a31b1e-ae7e-41f9-b9b0-22ab255a729b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:05:17.061114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.061099    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:17.062639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.062614    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952" pod "82a31b1e-ae7e-41f9-b9b0-22ab255a729b" (UID: "82a31b1e-ae7e-41f9-b9b0-22ab255a729b"). InnerVolumeSpecName "kube-api-access-bz952". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:17.163083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.163037    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz952\" (UniqueName: \"kubernetes.io/projected/82a31b1e-ae7e-41f9-b9b0-22ab255a729b-kube-api-access-bz952\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:17.449199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.449142    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 22:05:17.498220 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.498178    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-x54p7"]
Nov 02 22:05:17.597198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:17.597115    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" path="/var/lib/kubelet/pods/82a31b1e-ae7e-41f9-b9b0-22ab255a729b/volumes"
Nov 02 22:05:26.316616 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:26.316576    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:05:28.382893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.382854    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:05:28.383618 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.383599    5580 cpu_manager.go:414] "containerMap: removing container" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing"
Nov 02 22:05:28.383662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.383619    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing"
Nov 02 22:05:28.383753 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.383744    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="82a31b1e-ae7e-41f9-b9b0-22ab255a729b" containerName="payload-processing"
Nov 02 22:05:28.387482 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.387462    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.439486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.439449    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.439679 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.439640    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.439805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.439786    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.439928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.439907    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj6bb\" (UniqueName: \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.490343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.490309    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-3eb190-serving-cert\""
Nov 02 22:05:28.494049 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.494012    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:05:28.542012 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.541950    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.542183 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:28.542087    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-serving-cert: secret "maas-api-e2e-shared-b-w2-3eb190-serving-cert" not found
Nov 02 22:05:28.542183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.542160    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.542183 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:28.542168    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls podName:d7a9917c-8f7f-4539-ae45-82f912612c5a nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.042152083 +0000 UTC m=+1827.558647835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls") pod "maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" (UID: "d7a9917c-8f7f-4539-ae45-82f912612c5a") : secret "maas-api-e2e-shared-b-w2-3eb190-serving-cert" not found
Nov 02 22:05:28.542405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.542280    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj6bb\" (UniqueName: \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.542405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.542371    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.542908 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.542886    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.544744 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.544718    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:28.599546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:28.599512    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj6bb\" (UniqueName: \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:29.055093 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:29.055036    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:29.057853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:29.057819    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:29.301442 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:29.301402    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:29.682281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:29.682246    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:05:29.683061 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:05:29.683032    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a9917c_8f7f_4539_ae45_82f912612c5a.slice/crio-cb8ab8c4b8b40ac13165049a27883b79e79a7bd19749d0fd82b6c3d0ef6eb9e6 WatchSource:0}: Error finding container cb8ab8c4b8b40ac13165049a27883b79e79a7bd19749d0fd82b6c3d0ef6eb9e6: Status 404 returned error can't find the container with id cb8ab8c4b8b40ac13165049a27883b79e79a7bd19749d0fd82b6c3d0ef6eb9e6
Nov 02 22:05:30.136897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:30.136806    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" event={"ID":"d7a9917c-8f7f-4539-ae45-82f912612c5a","Type":"ContainerStarted","Data":"cb8ab8c4b8b40ac13165049a27883b79e79a7bd19749d0fd82b6c3d0ef6eb9e6"}
Nov 02 22:05:31.155504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:31.155462    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" event={"ID":"d7a9917c-8f7f-4539-ae45-82f912612c5a","Type":"ContainerStarted","Data":"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"}
Nov 02 22:05:31.156024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:31.155731    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:31.269617 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:31.269558    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" podStartSLOduration=2.458008596 podStartE2EDuration="3.269544322s" totalImagesPullingTime="811.535726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:31.26868313 +0000 UTC m=+1829.785178904" watchObservedRunningTime="2026-11-02 22:05:31.269544322 +0000 UTC m=+1829.786040094"
Nov 02 22:05:37.168286 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:37.168246    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:05:45.667816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:45.667776    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 22:05:45.668368 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:45.668052    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" containerID="cri-o://6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99" gracePeriod=30
Nov 02 22:05:49.627838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:49.627792    5580 patch_prober.go:29] interesting pod/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:05:49.628259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:49.627853    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:05:51.024236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.024185    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:05:51.043489 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.043457    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 22:05:51.118938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.118890    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119101    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119162    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119208    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119253    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119396    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:51.119549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119423    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:51.119549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119411    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119511    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:51.119838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119557    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119682    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119785    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\") pod \"4143753d-9a95-487b-81fc-040c44d38c63\" (UID: \"4143753d-9a95-487b-81fc-040c44d38c63\") "
Nov 02 22:05:51.119999 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.119950    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:05:51.120356 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120328    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:51.120725 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120702    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4143753d-9a95-487b-81fc-040c44d38c63-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.120835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120729    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.120835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120743    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.120835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120756    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.120835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.120769    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.121603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.121578    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "kube-api-access-tcwvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:51.121786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.121764    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:05:51.121871 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.121848    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:05:51.122350 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.122331    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token" pod "4143753d-9a95-487b-81fc-040c44d38c63" (UID: "4143753d-9a95-487b-81fc-040c44d38c63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:05:51.222985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.222927    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4143753d-9a95-487b-81fc-040c44d38c63-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.222985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.222984    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4143753d-9a95-487b-81fc-040c44d38c63-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.223255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.222999    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.223255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.223013    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcwvd\" (UniqueName: \"kubernetes.io/projected/4143753d-9a95-487b-81fc-040c44d38c63-kube-api-access-tcwvd\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:05:51.299423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.299330    5580 generic.go:356] "Generic (PLEG): container finished" podID="4143753d-9a95-487b-81fc-040c44d38c63" containerID="6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99" exitCode=0
Nov 02 22:05:51.299423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.299393    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"
Nov 02 22:05:51.299686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.299415    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" event={"ID":"4143753d-9a95-487b-81fc-040c44d38c63","Type":"ContainerDied","Data":"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"}
Nov 02 22:05:51.299686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.299480    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b" event={"ID":"4143753d-9a95-487b-81fc-040c44d38c63","Type":"ContainerDied","Data":"64438b1ddb080eef599c6a2fd1924e67a6d9bab7934f7e43092619be7acdbcc2"}
Nov 02 22:05:51.299686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.299509    5580 scope.go:122] "RemoveContainer" containerID="6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"
Nov 02 22:05:51.312061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.312036    5580 scope.go:122] "RemoveContainer" containerID="6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"
Nov 02 22:05:51.312356 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:51.312336    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99\": container with ID starting with 6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99 not found: ID does not exist" containerID="6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"
Nov 02 22:05:51.312404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.312366    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99"} err="failed to get container status \"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99\": rpc error: code = NotFound desc = could not find container \"6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99\": container with ID starting with 6cd9ed14ce31237978552dddb6aa83263d2fb321f733074e5a83934b488b2b99 not found: ID does not exist"
Nov 02 22:05:51.500980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.500001    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 22:05:51.509321 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.509281    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-fs78b"]
Nov 02 22:05:51.607190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:51.607097    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4143753d-9a95-487b-81fc-040c44d38c63" path="/var/lib/kubelet/pods/4143753d-9a95-487b-81fc-040c44d38c63/volumes"
Nov 02 22:05:53.917131 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:53.917076    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.055503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.055456    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.266992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.266074    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.266992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.266358    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.266992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.266731    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:05:54.326114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.325225    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.374986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.363070    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.465116 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.465061    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.493117 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.493067    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.572757 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.572648    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:54.928105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.928051    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 22:05:54.929555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:54.929527    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 22:05:55.064106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.064056    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.064877 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.064851    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 22:05:55.274067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.266597    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 22:05:55.274067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.267003    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 22:05:55.274067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.269060    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.274067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.269396    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 22:05:55.282352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.282235    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Nov 02 22:05:55.352988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.348047    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.397992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.386360    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 22:05:55.451032 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.441017    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.489982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.451908    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 22:05:55.528039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.493364    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.528039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.493985    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 22:05:55.528039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.495917    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:55.528039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:55.509171    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 22:05:56.147982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.112394    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.147982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.125005    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.302260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.302199    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 22:05:56.336031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.335982    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.352178 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.352116    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.357123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.357041    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:05:56.498013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.497655    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.498367 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.498324    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.505981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.498763    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.517993 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.516285    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.521125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.521073    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 02 22:05:56.636986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:56.611281    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:56.689309 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:05:56.688484    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:05:57.127978 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.127122    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.302687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.302619    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.369982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.340499    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.373211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.370456    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.374391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.370633    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.507109 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.507062    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.507922 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.507402    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.509206 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.507437    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 22:05:57.509703 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.508161    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 22:05:57.529307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.529277    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 22:05:57.531006 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.527511    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.531724 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.527017    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:57.571443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.568045    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:05:57.961589 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:57.961533    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:58.145231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.145187    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.339713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.339608    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:58.376080 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.376034    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.376712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.376683    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.377200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.377178    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.407382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.407352    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 22:05:58.428994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.428941    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 22:05:58.510677 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.510067    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.522106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.522056    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:58.555975 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.552467    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:58.557930 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.557885    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:05:58.568422 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.568368    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:05:58.569320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.569298    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 22:05:58.652525 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.652436    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 22:05:58.694435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.677232    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 22:05:58.694435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.677301    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 22:05:58.741039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.741007    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 22:05:58.746275 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.746252    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 22:05:58.746678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.746656    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc"
Nov 02 22:05:58.749889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.749675    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 22:05:58.750196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.750179    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 22:05:58.751216 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:05:58.751200    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 22:06:04.008913 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:04.008875    5580 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3762c23d_3be1_47c7_bfda_024fd935d42e.slice/crio-caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda.scope\": RecentStats: unable to find data in memory cache], [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@6-28751-0.service\": RecentStats: unable to find data in memory cache]"
Nov 02 22:06:04.195451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.195414    5580 generic.go:356] "Generic (PLEG): container finished" podID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerID="caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda" exitCode=0
Nov 02 22:06:04.195653 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.195486    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerDied","Data":"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"}
Nov 02 22:06:04.196013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.195991    5580 scope.go:122] "RemoveContainer" containerID="caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"
Nov 02 22:06:04.197620 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.197596    5580 generic.go:356] "Generic (PLEG): container finished" podID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerID="64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4" exitCode=0
Nov 02 22:06:04.197736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.197673    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerDied","Data":"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"}
Nov 02 22:06:04.198039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.198005    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:04.198127 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.198105    5580 scope.go:122] "RemoveContainer" containerID="64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"
Nov 02 22:06:04.198193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.198103    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:04.198257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.198239    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:04.199841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.199812    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:04.199968 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.199875    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:04.199968 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:04.199909    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:05.215800 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.215752    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerStarted","Data":"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"}
Nov 02 22:06:05.221406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.220789    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerStarted","Data":"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"}
Nov 02 22:06:05.303980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.296445    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:06:05.303980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.296571    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:05.303980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.296615    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:05.373139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.373109    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:06:05.461326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.461285    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:06:05.462732 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.462702    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:05.462884 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:05.462750    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:06.299994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:06.299456    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:06.299994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:06.299532    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:06.461104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:06.461067    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:06.461365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:06.461336    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:07.307983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.303825    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:07.307983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.303888    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:07.321406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.320425    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:06:07.321620 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.321592    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy"
Nov 02 22:06:07.321620 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.321614    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy"
Nov 02 22:06:07.328984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.321804    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4143753d-9a95-487b-81fc-040c44d38c63" containerName="istio-proxy"
Nov 02 22:06:07.331773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.331747    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.349299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.349262    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:06:07.369889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.369851    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.370119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.369898    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.370119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.370058    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.370236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.370202    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpkfm\" (UniqueName: \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.400982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.400302    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-224f53-serving-cert\""
Nov 02 22:06:07.461429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.461397    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:07.461686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.461455    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:07.472352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.472304    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.472614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.472376    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.472614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.472470    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.472614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.472606    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpkfm\" (UniqueName: \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.473210 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:07.473183    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-serving-cert: secret "maas-api-e2e-api-a-224f53-serving-cert" not found
Nov 02 22:06:07.473885 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:07.473862    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls podName:5316cc5e-1937-4249-9d61-0419fbaca216 nodeName:}" failed. No retries permitted until 2026-11-02 22:06:07.973839977 +0000 UTC m=+1866.490335753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls") pod "maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" (UID: "5316cc5e-1937-4249-9d61-0419fbaca216") : secret "maas-api-e2e-api-a-224f53-serving-cert" not found
Nov 02 22:06:07.474270 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.474245    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.475732 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.475709    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:07.566376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:07.563747    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpkfm\" (UniqueName: \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:08.021084 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.020416    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:08.026534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.026317    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") pod \"maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:08.251191 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.251152    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:08.329736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.329037    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:08.329736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.329097    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:08.460770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.460738    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:08.460968 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.460819    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:08.734182 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:08.734115    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:06:09.252702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:09.252664    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" event={"ID":"5316cc5e-1937-4249-9d61-0419fbaca216","Type":"ContainerStarted","Data":"9651c01b13f2630d247d8ddc8af68d8ad7e4b4e8b6852825c40592170218d896"}
Nov 02 22:06:09.306979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:09.306430    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:09.306979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:09.306497    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:09.462990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:09.462199    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:09.462990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:09.462303    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:10.261602 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.261564    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" event={"ID":"5316cc5e-1937-4249-9d61-0419fbaca216","Type":"ContainerStarted","Data":"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"}
Nov 02 22:06:10.261881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.261867    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:10.293982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.293443    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:06:10.293982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.293516    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 02 22:06:10.384985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.375877    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" podStartSLOduration=2.463702907 podStartE2EDuration="3.375860808s" totalImagesPullingTime="912.157901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:06:10.374034994 +0000 UTC m=+1868.890530766" watchObservedRunningTime="2026-11-02 22:06:10.375860808 +0000 UTC m=+1868.892356580"
Nov 02 22:06:10.461971 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.461904    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Nov 02 22:06:10.462184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:10.462018    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Nov 02 22:06:11.181061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:11.181017    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:06:11.299380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:11.299348    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:06:11.300667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:11.300642    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:06:11.467341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:11.467256    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:06:11.468415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:11.468389    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:06:16.273850 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:16.273815    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:06:26.313567 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:26.313522    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:06:34.480111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:34.480069    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:06:45.556590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.556491    5580 generic.go:356] "Generic (PLEG): container finished" podID="d26ff062-e4ec-4516-9691-799cf88e4626" containerID="91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66" exitCode=0
Nov 02 22:06:45.556590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.556567    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerDied","Data":"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"}
Nov 02 22:06:45.557181 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.557164    5580 scope.go:122] "RemoveContainer" containerID="91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"
Nov 02 22:06:45.559175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.559147    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:45.559275 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.559216    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:45.559275 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:45.559248    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:46.571301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:46.571264    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerStarted","Data":"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"}
Nov 02 22:06:47.307210 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:47.307176    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:06:47.307400 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:47.307377    5580 patch_prober.go:29] interesting pod/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 02 22:06:47.307449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:47.307426    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 02 22:06:48.307085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:48.307041    5580 patch_prober.go:29] interesting pod/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 02 22:06:48.307487 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:48.307102    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 02 22:06:49.306535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:49.306492    5580 patch_prober.go:29] interesting pod/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 02 22:06:49.306768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:49.306561    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 02 22:06:50.106135 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.106092    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:06:50.106600 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.106355    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" podUID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" containerName="maas-api" containerID="cri-o://e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b" gracePeriod=30
Nov 02 22:06:50.307776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.307741    5580 patch_prober.go:29] interesting pod/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 02 22:06:50.308153 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.308105    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 02 22:06:50.464287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.464239    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:06:50.577502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.575531    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\") pod \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") "
Nov 02 22:06:50.577502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.575596    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\" (UniqueName: \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\") pod \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") "
Nov 02 22:06:50.577502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.575817    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\") pod \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") "
Nov 02 22:06:50.577502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.576080    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") pod \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\" (UID: \"ca2f7793-b888-4eed-bed4-71e8eee7c60d\") "
Nov 02 22:06:50.580166 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.580118    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca" pod "ca2f7793-b888-4eed-bed4-71e8eee7c60d" (UID: "ca2f7793-b888-4eed-bed4-71e8eee7c60d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:50.650031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.640504    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq" pod "ca2f7793-b888-4eed-bed4-71e8eee7c60d" (UID: "ca2f7793-b888-4eed-bed4-71e8eee7c60d"). InnerVolumeSpecName "kube-api-access-fdnpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:50.650031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.644536    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs" pod "ca2f7793-b888-4eed-bed4-71e8eee7c60d" (UID: "ca2f7793-b888-4eed-bed4-71e8eee7c60d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:50.650031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.649505    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls" pod "ca2f7793-b888-4eed-bed4-71e8eee7c60d" (UID: "ca2f7793-b888-4eed-bed4-71e8eee7c60d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:50.660548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.660140    5580 generic.go:356] "Generic (PLEG): container finished" podID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" containerID="e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b" exitCode=0
Nov 02 22:06:50.660548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.660350    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" event={"ID":"ca2f7793-b888-4eed-bed4-71e8eee7c60d","Type":"ContainerDied","Data":"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"}
Nov 02 22:06:50.660548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.660391    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9" event={"ID":"ca2f7793-b888-4eed-bed4-71e8eee7c60d","Type":"ContainerDied","Data":"460c0b6cc5a4369bc2867fd0364ad0caad607c84b2df7f5dd1344382835c8902"}
Nov 02 22:06:50.660548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.660411    5580 scope.go:122] "RemoveContainer" containerID="e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"
Nov 02 22:06:50.660548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.660548    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"
Nov 02 22:06:50.710609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.709880    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2f7793-b888-4eed-bed4-71e8eee7c60d-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:06:50.710609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.709915    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:06:50.710609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.709930    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2f7793-b888-4eed-bed4-71e8eee7c60d-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:06:50.710609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.709944    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdnpq\" (UniqueName: \"kubernetes.io/projected/ca2f7793-b888-4eed-bed4-71e8eee7c60d-kube-api-access-fdnpq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:06:50.711485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.711447    5580 scope.go:122] "RemoveContainer" containerID="e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"
Nov 02 22:06:50.717202 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:50.714677    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b\": container with ID starting with e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b not found: ID does not exist" containerID="e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"
Nov 02 22:06:50.717202 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.714735    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b"} err="failed to get container status \"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b\": rpc error: code = NotFound desc = could not find container \"e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b\": container with ID starting with e22e8df87675862e438c6758b08710e722df08a296c3f293b20ac5aa15ac880b not found: ID does not exist"
Nov 02 22:06:50.826068 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.826012    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:06:50.831255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:50.831217    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2be7cc3-65f849d6c6-szjm9"]
Nov 02 22:06:51.307230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:51.307191    5580 patch_prober.go:29] interesting pod/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 02 22:06:51.307705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:51.307258    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 02 22:06:51.595791 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:51.595760    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" path="/var/lib/kubelet/pods/ca2f7793-b888-4eed-bed4-71e8eee7c60d/volumes"
Nov 02 22:06:52.312184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:52.312153    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:06:52.312651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:52.312476    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:06:52.313274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:52.313255    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:06:55.682988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.668114    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:06:55.682988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.669170    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" containerName="maas-api"
Nov 02 22:06:55.682988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.669196    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" containerName="maas-api"
Nov 02 22:06:55.682988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.669442    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca2f7793-b888-4eed-bed4-71e8eee7c60d" containerName="maas-api"
Nov 02 22:06:55.682988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.675372    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.883173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.883142    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-588b5c-openshift-default-dockercfg-s4642\""
Nov 02 22:06:55.894502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.894363    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:06:55.912143 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912106    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912185    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrf72\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912227    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912255    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912427    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912484    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912520    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912676 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912572    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:55.912747 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:55.912728    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.014995    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015071    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015100    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015143    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015285    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015408    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015489 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015459    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015547 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015472    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrf72\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015541    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015574    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.015819 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.015794    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.016066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.016017    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.016310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.016287    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.016384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.016345    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.018191 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.018161    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.018564 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.018548    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.073002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.072418    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrf72\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.073002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.072436    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\") pod \"e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") " pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.314716 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:06:56.314613    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:06:56.320096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.320052    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:56.758604 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:06:56.758567    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9959ae_f7d0_48c3_8370_3647e4ca50e1.slice/crio-4c217ab78eacd5b63cb0942cf2cd1272e28f20f31df97edeeb38f717d69d1330 WatchSource:0}: Error finding container 4c217ab78eacd5b63cb0942cf2cd1272e28f20f31df97edeeb38f717d69d1330: Status 404 returned error can't find the container with id 4c217ab78eacd5b63cb0942cf2cd1272e28f20f31df97edeeb38f717d69d1330
Nov 02 22:06:56.759515 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.759484    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:06:56.761678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.761640    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:56.761780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.761714    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:56.761780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:56.761746    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:06:57.754427 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:57.754387    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerStarted","Data":"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"}
Nov 02 22:06:57.754637 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:57.754446    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerStarted","Data":"4c217ab78eacd5b63cb0942cf2cd1272e28f20f31df97edeeb38f717d69d1330"}
Nov 02 22:06:57.921024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:57.920946    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podStartSLOduration=2.9209301979999998 podStartE2EDuration="2.920930198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:06:57.919341783 +0000 UTC m=+1916.435837557" watchObservedRunningTime="2026-11-02 22:06:57.920930198 +0000 UTC m=+1916.437425970"
Nov 02 22:06:58.321116 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:58.321069    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:58.327374 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:58.327343    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:58.762267 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:58.762239    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:06:58.763212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:06:58.763190    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:07:02.679433 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.679387    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:07:02.708604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.708571    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.785358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.785041    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqhlq\" (UniqueName: \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.785358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.785158    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.798979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.790823    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:07:02.865807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.865772    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e2be7cc3\""
Nov 02 22:07:02.880891 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.880853    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:07:02.892104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.885385    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:02.895118 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.887154    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqhlq\" (UniqueName: \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.896321 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.896294    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.897474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.897446    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.915582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.915537    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:07:02.946141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.946064    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqhlq\" (UniqueName: \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\") pod \"payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:02.949221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.949192    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e2be7cc3-dockercfg-bxspk\""
Nov 02 22:07:02.998323 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.998283    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk25t\" (UniqueName: \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:02.998526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:02.998341    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.046244 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.046211    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:03.099785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.099746    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk25t\" (UniqueName: \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.100088 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.099805    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.100700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.100672    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.166915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.165401    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk25t\" (UniqueName: \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\") pod \"payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") " pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.233265 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.233178    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:03.434541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.434494    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:07:03.438897 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:07:03.438804    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7e38bd_99b1_4b0e_a037_0fc95bc4a758.slice/crio-b31109be5a3ead69d99bb655e5393b8a71a4fabad9f659048e1070aafb0f464d WatchSource:0}: Error finding container b31109be5a3ead69d99bb655e5393b8a71a4fabad9f659048e1070aafb0f464d: Status 404 returned error can't find the container with id b31109be5a3ead69d99bb655e5393b8a71a4fabad9f659048e1070aafb0f464d
Nov 02 22:07:03.640340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.640311    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:07:03.641318 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:07:03.641294    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771f085d_b1f8_4afd_b4ba_e5c4f60dcf9d.slice/crio-d443802a426684baeb299401d113cfd8a286883505df73c846d37c7899bdbcf6 WatchSource:0}: Error finding container d443802a426684baeb299401d113cfd8a286883505df73c846d37c7899bdbcf6: Status 404 returned error can't find the container with id d443802a426684baeb299401d113cfd8a286883505df73c846d37c7899bdbcf6
Nov 02 22:07:03.919400 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.919348    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" event={"ID":"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758","Type":"ContainerStarted","Data":"50b5639bafb974a233495cf03d1471a633f6b815eb19daf52c6e11c7d5fb6334"}
Nov 02 22:07:03.919834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.919455    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" event={"ID":"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758","Type":"ContainerStarted","Data":"b31109be5a3ead69d99bb655e5393b8a71a4fabad9f659048e1070aafb0f464d"}
Nov 02 22:07:03.919834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.919487    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:03.922044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.922007    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" event={"ID":"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d","Type":"ContainerStarted","Data":"c546a7d84c55507f8b836ffd7e5ec4d08ce7ec8da45847fee15732cccb4d6d9a"}
Nov 02 22:07:03.922196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.922072    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" event={"ID":"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d","Type":"ContainerStarted","Data":"d443802a426684baeb299401d113cfd8a286883505df73c846d37c7899bdbcf6"}
Nov 02 22:07:03.922196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:03.922130    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:04.034234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:04.034171    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" podStartSLOduration=2.03415661 podStartE2EDuration="2.03415661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:07:04.032768929 +0000 UTC m=+1922.549264701" watchObservedRunningTime="2026-11-02 22:07:04.03415661 +0000 UTC m=+1922.550652770"
Nov 02 22:07:04.170295 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:04.170059    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" podStartSLOduration=2.170033795 podStartE2EDuration="2.170033795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:07:04.168616171 +0000 UTC m=+1922.685111945" watchObservedRunningTime="2026-11-02 22:07:04.170033795 +0000 UTC m=+1922.686529568"
Nov 02 22:07:07.826687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:07.826642    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:07:07.836980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:07.834262    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:07.888663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:07.888630    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-588b5c-dockercfg-gpmtv\""
Nov 02 22:07:07.897046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:07.894472    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-588b5c\""
Nov 02 22:07:08.043004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:07.909851    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:07:08.055077 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.055007    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:07:08.062047 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.062013    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.063419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.063387    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:07:08.075436 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.075396    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8624f\" (UniqueName: \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.075929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.075906    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.095168 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.095061    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djw4m\" (UniqueName: \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.109982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.102193    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.216000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.215935    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8624f\" (UniqueName: \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.216187 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.216174    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.216288 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.216211    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djw4m\" (UniqueName: \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.216358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.216307    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.217320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.217292    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.217320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.217303    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.272992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.272915    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djw4m\" (UniqueName: \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\") pod \"payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") " pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.272992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.272914    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8624f\" (UniqueName: \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\") pod \"payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.439832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.439796    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:08.540411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.540371    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:08.852797 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:07:08.852763    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24efedd7_9d0c_47b7_a6dc_7abf2347434b.slice/crio-01ccc825670f1b7cd0485672636ae47f3121684d3687e5ffa507062613dc6ba8 WatchSource:0}: Error finding container 01ccc825670f1b7cd0485672636ae47f3121684d3687e5ffa507062613dc6ba8: Status 404 returned error can't find the container with id 01ccc825670f1b7cd0485672636ae47f3121684d3687e5ffa507062613dc6ba8
Nov 02 22:07:08.853193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.852888    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:07:08.960866 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:08.960834    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:07:08.961915 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:07:08.961885    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ba0ad0_fee6_48a0_a360_75228b16c54d.slice/crio-567fb26b66943b0b9f3519de6077b452dac06fa2e3e76751985cda0d1593b502 WatchSource:0}: Error finding container 567fb26b66943b0b9f3519de6077b452dac06fa2e3e76751985cda0d1593b502: Status 404 returned error can't find the container with id 567fb26b66943b0b9f3519de6077b452dac06fa2e3e76751985cda0d1593b502
Nov 02 22:07:09.000129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:09.000092    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" event={"ID":"b1ba0ad0-fee6-48a0-a360-75228b16c54d","Type":"ContainerStarted","Data":"567fb26b66943b0b9f3519de6077b452dac06fa2e3e76751985cda0d1593b502"}
Nov 02 22:07:09.001831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:09.001798    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" event={"ID":"24efedd7-9d0c-47b7-a6dc-7abf2347434b","Type":"ContainerStarted","Data":"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"}
Nov 02 22:07:09.001987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:09.001853    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" event={"ID":"24efedd7-9d0c-47b7-a6dc-7abf2347434b","Type":"ContainerStarted","Data":"01ccc825670f1b7cd0485672636ae47f3121684d3687e5ffa507062613dc6ba8"}
Nov 02 22:07:09.001987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:09.001892    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:09.120323 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:09.120263    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" podStartSLOduration=2.120245836 podStartE2EDuration="2.120245836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:07:09.116798546 +0000 UTC m=+1927.633294319" watchObservedRunningTime="2026-11-02 22:07:09.120245836 +0000 UTC m=+1927.636741650"
Nov 02 22:07:10.011146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:10.011103    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" event={"ID":"b1ba0ad0-fee6-48a0-a360-75228b16c54d","Type":"ContainerStarted","Data":"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"}
Nov 02 22:07:10.011687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:10.011173    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:10.129673 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:10.129607    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" podStartSLOduration=3.129591595 podStartE2EDuration="3.129591595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:07:10.125063365 +0000 UTC m=+1928.641559138" watchObservedRunningTime="2026-11-02 22:07:10.129591595 +0000 UTC m=+1928.646087368"
Nov 02 22:07:13.037262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.037174    5580 generic.go:356] "Generic (PLEG): container finished" podID="4389b173-7d78-48c9-926f-065f5495ac7c" containerID="33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76" exitCode=0
Nov 02 22:07:13.037262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.037225    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerDied","Data":"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"}
Nov 02 22:07:13.037824 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.037757    5580 scope.go:122] "RemoveContainer" containerID="33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"
Nov 02 22:07:13.039984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.039936    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:07:13.040160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.040044    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:07:13.040160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:13.040117    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:07:14.046238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.046195    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerStarted","Data":"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"}
Nov 02 22:07:14.699983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.699572    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:07:14.700203 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.700061    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 02 22:07:14.700203 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.700122    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 02 22:07:14.937061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.934646    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:07:14.937061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:14.935917    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:07:15.702239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:15.702025    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 02 22:07:15.702239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:15.702104    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 02 22:07:16.700986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:16.700539    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 02 22:07:16.700986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:16.700614    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 02 22:07:17.699984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:17.695352    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 02 22:07:17.699984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:17.695416    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 02 22:07:18.694982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:18.688480    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 02 22:07:18.694982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:18.688561    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 02 22:07:19.692593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:19.692562    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:07:19.693195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:19.693078    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:07:19.693854 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:19.693831    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:07:20.014472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:20.014391    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:07:21.023414 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:21.023380    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:07:26.311656 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:07:26.311615    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:07:32.186881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:32.186841    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:07:35.272179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:35.272109    5580 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:07:35.328926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:35.328874    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 02 22:07:35.419149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:35.419093    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:07:35.526210 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:35.526093    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:07:45.687790 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:45.687747    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 22:07:45.688291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:45.688020    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy" containerID="cri-o://2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7" gracePeriod=30
Nov 02 22:07:50.974443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:50.974410    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 22:07:51.069237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069198    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069297    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069404    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069572 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069551    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069590    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069630    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:07:51.069687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069644    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.069687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069658    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069789    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069863    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069973 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069904    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\") pod \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\" (UID: \"79e73f74-8f30-4f2a-a24c-7625ec0b3247\") "
Nov 02 22:07:51.069973 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.069941    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.070170 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070131    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.070258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070229    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.070881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070839    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.070881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070871    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.071085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070887    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.071085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070905    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.071085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.070922    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.071989 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.071913    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:07:51.072245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.072219    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.072393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.072320    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "kube-api-access-k8gdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:07:51.072393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.072359    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token" pod "79e73f74-8f30-4f2a-a24c-7625ec0b3247" (UID: "79e73f74-8f30-4f2a-a24c-7625ec0b3247"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:07:51.172247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.172200    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8gdp\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-kube-api-access-k8gdp\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.172247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.172236    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.172247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.172246    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.172247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.172255    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79e73f74-8f30-4f2a-a24c-7625ec0b3247-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.202142 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.202057    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 22:07:51.202301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.202266    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerName="discovery" containerID="cri-o://2e7b55bbf3c0c7a223e5c104a4307fa85d96a2432d6ed250fa4248ed5ca285db" gracePeriod=30
Nov 02 22:07:51.418836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.418788    5580 generic.go:356] "Generic (PLEG): container finished" podID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerID="2e7b55bbf3c0c7a223e5c104a4307fa85d96a2432d6ed250fa4248ed5ca285db" exitCode=0
Nov 02 22:07:51.419051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.419016    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" event={"ID":"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7","Type":"ContainerDied","Data":"2e7b55bbf3c0c7a223e5c104a4307fa85d96a2432d6ed250fa4248ed5ca285db"}
Nov 02 22:07:51.420825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.420798    5580 generic.go:356] "Generic (PLEG): container finished" podID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerID="2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7" exitCode=0
Nov 02 22:07:51.421073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.420868    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" event={"ID":"79e73f74-8f30-4f2a-a24c-7625ec0b3247","Type":"ContainerDied","Data":"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"}
Nov 02 22:07:51.421272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.421245    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95" event={"ID":"79e73f74-8f30-4f2a-a24c-7625ec0b3247","Type":"ContainerDied","Data":"40cf1416ed1bee79b89660f2f8e1d82cacb30727b3097418244fc0136d0b10f1"}
Nov 02 22:07:51.421353 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.420917    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"
Nov 02 22:07:51.421353 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.421287    5580 scope.go:122] "RemoveContainer" containerID="2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"
Nov 02 22:07:51.443079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.443002    5580 scope.go:122] "RemoveContainer" containerID="2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"
Nov 02 22:07:51.443851 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:07:51.443789    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7\": container with ID starting with 2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7 not found: ID does not exist" containerID="2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"
Nov 02 22:07:51.444036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.443824    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7"} err="failed to get container status \"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7\": rpc error: code = NotFound desc = could not find container \"2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7\": container with ID starting with 2e9d9e849dca55b81be50104f0adcd433cd3898b276c8fe4335fae16c990cae7 not found: ID does not exist"
Nov 02 22:07:51.539358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.539330    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 22:07:51.609355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.609329    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 22:07:51.609504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.609367    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-g9g95"]
Nov 02 22:07:51.684553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684514    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.684553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684559    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.684773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684707    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.684823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684793    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.684910 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684863    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.684985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.684967    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.685355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.685144    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\") pod \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\" (UID: \"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7\") "
Nov 02 22:07:51.685649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.685620    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:07:51.685947 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.685924    5580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-ca-configmap\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.687349 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687291    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:07:51.687470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687411    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:07:51.687470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687423    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:07:51.687594 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687497    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "kube-api-access-4zrjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:07:51.687708 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687691    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:07:51.687927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.687904    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig" pod "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" (UID: "303c4f5a-fbfe-4fe7-b862-92c4dd8197c7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787496    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zrjk\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-kube-api-access-4zrjk\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787534    5580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-kubeconfig\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787544    5580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-local-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787552    5580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-csr-dns-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787562    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:51.787593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:51.787572    5580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7-cacerts\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:07:52.431234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:52.431208    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"
Nov 02 22:07:52.431706 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:52.431205    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh" event={"ID":"303c4f5a-fbfe-4fe7-b862-92c4dd8197c7","Type":"ContainerDied","Data":"a04b4d8714fd1cd475f6f2a969b786b0478f9d0aa1eb22f7e9646b56f53988ae"}
Nov 02 22:07:52.431706 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:52.431344    5580 scope.go:122] "RemoveContainer" containerID="2e7b55bbf3c0c7a223e5c104a4307fa85d96a2432d6ed250fa4248ed5ca285db"
Nov 02 22:07:52.598671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:52.598627    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 22:07:52.648819 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:52.648760    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdksh"]
Nov 02 22:07:53.595044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:53.595009    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" path="/var/lib/kubelet/pods/303c4f5a-fbfe-4fe7-b862-92c4dd8197c7/volumes"
Nov 02 22:07:53.595586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:53.595571    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" path="/var/lib/kubelet/pods/79e73f74-8f30-4f2a-a24c-7625ec0b3247/volumes"
Nov 02 22:07:56.313072 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:07:56.313033    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:07:59.687704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:07:59.687218    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:08:04.688307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:04.688222    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" podUID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" containerName="payload-pre-processing" containerID="cri-o://58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214" gracePeriod=25
Nov 02 22:08:04.966297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:04.966270    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:08:05.116945 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.116901    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\") pod \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") "
Nov 02 22:08:05.117178 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.117158    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\" (UniqueName: \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\") pod \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\" (UID: \"639de4ba-63b5-45a3-a72e-f88a676ffa5d\") "
Nov 02 22:08:05.117418 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.117391    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume" pod "639de4ba-63b5-45a3-a72e-f88a676ffa5d" (UID: "639de4ba-63b5-45a3-a72e-f88a676ffa5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:08:05.117882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.117859    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/639de4ba-63b5-45a3-a72e-f88a676ffa5d-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:05.119520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.119488    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc" pod "639de4ba-63b5-45a3-a72e-f88a676ffa5d" (UID: "639de4ba-63b5-45a3-a72e-f88a676ffa5d"). InnerVolumeSpecName "kube-api-access-kwmqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:08:05.219965 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.219858    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwmqc\" (UniqueName: \"kubernetes.io/projected/639de4ba-63b5-45a3-a72e-f88a676ffa5d-kube-api-access-kwmqc\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:05.550287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.550178    5580 generic.go:356] "Generic (PLEG): container finished" podID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" containerID="58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214" exitCode=0
Nov 02 22:08:05.550287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.550234    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"
Nov 02 22:08:05.550287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.550247    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" event={"ID":"639de4ba-63b5-45a3-a72e-f88a676ffa5d","Type":"ContainerDied","Data":"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"}
Nov 02 22:08:05.550542 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.550335    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs" event={"ID":"639de4ba-63b5-45a3-a72e-f88a676ffa5d","Type":"ContainerDied","Data":"bf99e10c75daf685d5f1eab4def8d40b6ca79da817cce347537b270251b5e23f"}
Nov 02 22:08:05.550542 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.550358    5580 scope.go:122] "RemoveContainer" containerID="58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"
Nov 02 22:08:05.562897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.562874    5580 scope.go:122] "RemoveContainer" containerID="58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"
Nov 02 22:08:05.563265 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:05.563246    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214\": container with ID starting with 58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214 not found: ID does not exist" containerID="58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"
Nov 02 22:08:05.563355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.563276    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214"} err="failed to get container status \"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214\": rpc error: code = NotFound desc = could not find container \"58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214\": container with ID starting with 58afd243b36e2155a2c917ab41d86a2e85a57aa843aabe7db9d534be9f166214 not found: ID does not exist"
Nov 02 22:08:05.713308 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.713270    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:08:05.762770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:05.762732    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-224f53-67fcf94855-qb5hs"]
Nov 02 22:08:07.595875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:07.595841    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" path="/var/lib/kubelet/pods/639de4ba-63b5-45a3-a72e-f88a676ffa5d/volumes"
Nov 02 22:08:09.065320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:09.065275    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:08:14.065701 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.065575    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" podUID="a2408876-6604-456f-bdb1-b939c1945905" containerName="payload-processing" containerID="cri-o://18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100" gracePeriod=25
Nov 02 22:08:14.371207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.371178    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:08:14.486849 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.486785    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\" (UniqueName: \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\") pod \"a2408876-6604-456f-bdb1-b939c1945905\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") "
Nov 02 22:08:14.487164 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.487139    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\") pod \"a2408876-6604-456f-bdb1-b939c1945905\" (UID: \"a2408876-6604-456f-bdb1-b939c1945905\") "
Nov 02 22:08:14.488343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.488310    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume" pod "a2408876-6604-456f-bdb1-b939c1945905" (UID: "a2408876-6604-456f-bdb1-b939c1945905"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:08:14.490431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.490400    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb" pod "a2408876-6604-456f-bdb1-b939c1945905" (UID: "a2408876-6604-456f-bdb1-b939c1945905"). InnerVolumeSpecName "kube-api-access-9w4qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:08:14.589803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.589704    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4qb\" (UniqueName: \"kubernetes.io/projected/a2408876-6604-456f-bdb1-b939c1945905-kube-api-access-9w4qb\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:14.589803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.589738    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2408876-6604-456f-bdb1-b939c1945905-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:14.629461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.629427    5580 generic.go:356] "Generic (PLEG): container finished" podID="a2408876-6604-456f-bdb1-b939c1945905" containerID="18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100" exitCode=0
Nov 02 22:08:14.629683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.629480    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"
Nov 02 22:08:14.629683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.629511    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" event={"ID":"a2408876-6604-456f-bdb1-b939c1945905","Type":"ContainerDied","Data":"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"}
Nov 02 22:08:14.629683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.629579    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk" event={"ID":"a2408876-6604-456f-bdb1-b939c1945905","Type":"ContainerDied","Data":"9f8e21e0343f8dc5ed2f4c615274754fb920d82dfc78e32f5eee004d16b20a5e"}
Nov 02 22:08:14.629683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.629610    5580 scope.go:122] "RemoveContainer" containerID="18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"
Nov 02 22:08:14.641925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.641896    5580 scope.go:122] "RemoveContainer" containerID="18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"
Nov 02 22:08:14.642360 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:14.642331    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100\": container with ID starting with 18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100 not found: ID does not exist" containerID="18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"
Nov 02 22:08:14.642467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.642370    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100"} err="failed to get container status \"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100\": rpc error: code = NotFound desc = could not find container \"18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100\": container with ID starting with 18f920ff0f3b45055bb536dd82ff85272d1043cd8928582f3e2b6830aefdd100 not found: ID does not exist"
Nov 02 22:08:14.793512 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.793477    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:08:14.796435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:14.796412    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4fc235-575db4d5d9-r5fsk"]
Nov 02 22:08:15.595406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:15.595372    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2408876-6604-456f-bdb1-b939c1945905" path="/var/lib/kubelet/pods/a2408876-6604-456f-bdb1-b939c1945905/volumes"
Nov 02 22:08:26.362500 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:26.362462    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:08:29.789405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.789368    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:08:29.792057 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792012    5580 cpu_manager.go:414] "containerMap: removing container" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerName="discovery"
Nov 02 22:08:29.792274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792260    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerName="discovery"
Nov 02 22:08:29.792372 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792362    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a2408876-6604-456f-bdb1-b939c1945905" containerName="payload-processing"
Nov 02 22:08:29.792467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792456    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2408876-6604-456f-bdb1-b939c1945905" containerName="payload-processing"
Nov 02 22:08:29.792603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792593    5580 cpu_manager.go:414] "containerMap: removing container" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy"
Nov 02 22:08:29.792695 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792685    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy"
Nov 02 22:08:29.792786 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792776    5580 cpu_manager.go:414] "containerMap: removing container" podUID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" containerName="payload-pre-processing"
Nov 02 22:08:29.792859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.792848    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" containerName="payload-pre-processing"
Nov 02 22:08:29.793201 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.793184    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="303c4f5a-fbfe-4fe7-b862-92c4dd8197c7" containerName="discovery"
Nov 02 22:08:29.794211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.794189    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2408876-6604-456f-bdb1-b939c1945905" containerName="payload-processing"
Nov 02 22:08:29.794372 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.794361    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="79e73f74-8f30-4f2a-a24c-7625ec0b3247" containerName="istio-proxy"
Nov 02 22:08:29.794465 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.794455    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="639de4ba-63b5-45a3-a72e-f88a676ffa5d" containerName="payload-pre-processing"
Nov 02 22:08:29.799599 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.799571    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:29.847018 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.846951    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:08:29.851449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.851416    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-17435d-dockercfg-nhn48\""
Nov 02 22:08:29.890563 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.890531    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-17435d\""
Nov 02 22:08:29.912745 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.912683    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:29.913006 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:29.912941    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c886b\" (UniqueName: \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.014106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.014066    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c886b\" (UniqueName: \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.014289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.014163    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.014857 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.014837    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.070087 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.070010    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c886b\" (UniqueName: \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\") pod \"payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.117986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.117911    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:30.497576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.497547    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:08:30.499110 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:08:30.499081    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b9229a_f32d_4b46_9247_4f9d32a6bea6.slice/crio-e5d9ae6328f3e733af8c1c414a4f0308f0540a1f39629072da0513d138f9e6dd WatchSource:0}: Error finding container e5d9ae6328f3e733af8c1c414a4f0308f0540a1f39629072da0513d138f9e6dd: Status 404 returned error can't find the container with id e5d9ae6328f3e733af8c1c414a4f0308f0540a1f39629072da0513d138f9e6dd
Nov 02 22:08:30.903344 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.903248    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" event={"ID":"72b9229a-f32d-4b46-9247-4f9d32a6bea6","Type":"ContainerStarted","Data":"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"}
Nov 02 22:08:30.903344 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.903322    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" event={"ID":"72b9229a-f32d-4b46-9247-4f9d32a6bea6","Type":"ContainerStarted","Data":"e5d9ae6328f3e733af8c1c414a4f0308f0540a1f39629072da0513d138f9e6dd"}
Nov 02 22:08:30.903865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:30.903474    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:31.024415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:31.022203    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" podStartSLOduration=2.022184792 podStartE2EDuration="2.022184792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:08:31.018410385 +0000 UTC m=+2009.534906159" watchObservedRunningTime="2026-11-02 22:08:31.022184792 +0000 UTC m=+2009.538680564"
Nov 02 22:08:41.049275 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:41.049236    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:08:41.915085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:41.915054    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:08:42.828639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:42.828602    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:08:43.300264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.300156    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:08:43.300596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.300564    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" podUID="d7a9917c-8f7f-4539-ae45-82f912612c5a" containerName="maas-api" containerID="cri-o://b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a" gracePeriod=30
Nov 02 22:08:43.649925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.649890    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:08:43.757546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.757508    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\") pod \"d7a9917c-8f7f-4539-ae45-82f912612c5a\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") "
Nov 02 22:08:43.757739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.757577    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\") pod \"d7a9917c-8f7f-4539-ae45-82f912612c5a\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") "
Nov 02 22:08:43.757739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.757681    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") pod \"d7a9917c-8f7f-4539-ae45-82f912612c5a\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") "
Nov 02 22:08:43.757739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.757726    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\" (UniqueName: \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\") pod \"d7a9917c-8f7f-4539-ae45-82f912612c5a\" (UID: \"d7a9917c-8f7f-4539-ae45-82f912612c5a\") "
Nov 02 22:08:43.760022 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.759987    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca" pod "d7a9917c-8f7f-4539-ae45-82f912612c5a" (UID: "d7a9917c-8f7f-4539-ae45-82f912612c5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:08:43.761304 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.761273    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls" pod "d7a9917c-8f7f-4539-ae45-82f912612c5a" (UID: "d7a9917c-8f7f-4539-ae45-82f912612c5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:08:43.761446 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.761393    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs" pod "d7a9917c-8f7f-4539-ae45-82f912612c5a" (UID: "d7a9917c-8f7f-4539-ae45-82f912612c5a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:08:43.762179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.762151    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb" pod "d7a9917c-8f7f-4539-ae45-82f912612c5a" (UID: "d7a9917c-8f7f-4539-ae45-82f912612c5a"). InnerVolumeSpecName "kube-api-access-sj6bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:08:43.859823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.859728    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a9917c-8f7f-4539-ae45-82f912612c5a-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:43.859823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.859768    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:43.859823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.859777    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7a9917c-8f7f-4539-ae45-82f912612c5a-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:43.859823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:43.859785    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj6bb\" (UniqueName: \"kubernetes.io/projected/d7a9917c-8f7f-4539-ae45-82f912612c5a-kube-api-access-sj6bb\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:44.052436 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.052401    5580 generic.go:356] "Generic (PLEG): container finished" podID="d7a9917c-8f7f-4539-ae45-82f912612c5a" containerID="b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a" exitCode=0
Nov 02 22:08:44.052633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.052471    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"
Nov 02 22:08:44.052633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.052493    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" event={"ID":"d7a9917c-8f7f-4539-ae45-82f912612c5a","Type":"ContainerDied","Data":"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"}
Nov 02 22:08:44.052633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.052565    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9" event={"ID":"d7a9917c-8f7f-4539-ae45-82f912612c5a","Type":"ContainerDied","Data":"cb8ab8c4b8b40ac13165049a27883b79e79a7bd19749d0fd82b6c3d0ef6eb9e6"}
Nov 02 22:08:44.052633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.052595    5580 scope.go:122] "RemoveContainer" containerID="b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"
Nov 02 22:08:44.065714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.065684    5580 scope.go:122] "RemoveContainer" containerID="b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"
Nov 02 22:08:44.066201 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:44.066157    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a\": container with ID starting with b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a not found: ID does not exist" containerID="b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"
Nov 02 22:08:44.066380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.066197    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a"} err="failed to get container status \"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a\": rpc error: code = NotFound desc = could not find container \"b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a\": container with ID starting with b8ce0343dd35789e477ec1c91e6a6667494f6ae31e9add90be80c4cfcc33ad8a not found: ID does not exist"
Nov 02 22:08:44.215744 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.215700    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:08:44.220932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:44.220899    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-3eb190-945ffdbdc-l5qx9"]
Nov 02 22:08:45.595152 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:45.595115    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7a9917c-8f7f-4539-ae45-82f912612c5a" path="/var/lib/kubelet/pods/d7a9917c-8f7f-4539-ae45-82f912612c5a/volumes"
Nov 02 22:08:47.829127 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:47.829080    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" podUID="ebdb250f-d4bb-46cf-b337-266445d6251e" containerName="payload-processing" containerID="cri-o://c4808bbdcd05b9851660323a85855ff2d4b7feb333a6fa3a07a43fd4cd90a6a3" gracePeriod=25
Nov 02 22:08:48.083745 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.083665    5580 generic.go:356] "Generic (PLEG): container finished" podID="ebdb250f-d4bb-46cf-b337-266445d6251e" containerID="c4808bbdcd05b9851660323a85855ff2d4b7feb333a6fa3a07a43fd4cd90a6a3" exitCode=0
Nov 02 22:08:48.083912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.083738    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" event={"ID":"ebdb250f-d4bb-46cf-b337-266445d6251e","Type":"ContainerDied","Data":"c4808bbdcd05b9851660323a85855ff2d4b7feb333a6fa3a07a43fd4cd90a6a3"}
Nov 02 22:08:48.109865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.109834    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:08:48.124161 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.124126    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\") pod \"ebdb250f-d4bb-46cf-b337-266445d6251e\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") "
Nov 02 22:08:48.124321 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.124307    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\" (UniqueName: \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\") pod \"ebdb250f-d4bb-46cf-b337-266445d6251e\" (UID: \"ebdb250f-d4bb-46cf-b337-266445d6251e\") "
Nov 02 22:08:48.124507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.124484    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume" pod "ebdb250f-d4bb-46cf-b337-266445d6251e" (UID: "ebdb250f-d4bb-46cf-b337-266445d6251e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:08:48.124768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.124749    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdb250f-d4bb-46cf-b337-266445d6251e-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:48.126527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.126502    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z" pod "ebdb250f-d4bb-46cf-b337-266445d6251e" (UID: "ebdb250f-d4bb-46cf-b337-266445d6251e"). InnerVolumeSpecName "kube-api-access-tww6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:08:48.226671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:48.226630    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tww6z\" (UniqueName: \"kubernetes.io/projected/ebdb250f-d4bb-46cf-b337-266445d6251e-kube-api-access-tww6z\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:08:49.091965 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.091936    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"
Nov 02 22:08:49.092469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.091986    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml" event={"ID":"ebdb250f-d4bb-46cf-b337-266445d6251e","Type":"ContainerDied","Data":"d7f5ddf4d7b1da7f8359d0fcdc1ad3dba69c03ce70f7eadec40c1bf5c66e4e99"}
Nov 02 22:08:49.092469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.092061    5580 scope.go:122] "RemoveContainer" containerID="c4808bbdcd05b9851660323a85855ff2d4b7feb333a6fa3a07a43fd4cd90a6a3"
Nov 02 22:08:49.257217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.257177    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:08:49.306444 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.306397    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3eb190-5c65b698cf-68sml"]
Nov 02 22:08:49.595380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:49.595339    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebdb250f-d4bb-46cf-b337-266445d6251e" path="/var/lib/kubelet/pods/ebdb250f-d4bb-46cf-b337-266445d6251e/volumes"
Nov 02 22:08:54.774148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774111    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:08:54.774799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774783    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d7a9917c-8f7f-4539-ae45-82f912612c5a" containerName="maas-api"
Nov 02 22:08:54.774799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774800    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7a9917c-8f7f-4539-ae45-82f912612c5a" containerName="maas-api"
Nov 02 22:08:54.774876 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774830    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ebdb250f-d4bb-46cf-b337-266445d6251e" containerName="payload-processing"
Nov 02 22:08:54.774876 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774836    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebdb250f-d4bb-46cf-b337-266445d6251e" containerName="payload-processing"
Nov 02 22:08:54.774990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774971    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebdb250f-d4bb-46cf-b337-266445d6251e" containerName="payload-processing"
Nov 02 22:08:54.775033 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.774991    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7a9917c-8f7f-4539-ae45-82f912612c5a" containerName="maas-api"
Nov 02 22:08:54.777748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.777728    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:54.880486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.880443    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2050f3-serving-cert\""
Nov 02 22:08:54.884039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.884005    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:08:54.931281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.931241    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:54.931587 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.931550    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:54.931714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.931640    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j9ht\" (UniqueName: \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:54.932036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:54.932007    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.037192 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.037078    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.037192 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.037172    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.037621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.037238    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j9ht\" (UniqueName: \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.037621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.037551    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.040305 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.038555    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.042237 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:55.042205    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-serving-cert: secret "maas-api-e2e-inf-a-2050f3-serving-cert" not found
Nov 02 22:08:55.042403 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:55.042316    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls podName:c6bf5b71-fcec-406b-be53-9edacc5fad53 nodeName:}" failed. No retries permitted until 2026-11-02 22:08:55.542282035 +0000 UTC m=+2034.058777803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls") pod "maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" (UID: "c6bf5b71-fcec-406b-be53-9edacc5fad53") : secret "maas-api-e2e-inf-a-2050f3-serving-cert" not found
Nov 02 22:08:55.045729 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.045695    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.098659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.098620    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j9ht\" (UniqueName: \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.544889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.544847    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.547441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.547408    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") pod \"maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:55.691694 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:55.691660    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:56.068775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:56.068736    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:08:56.070107 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:08:56.070077    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bf5b71_fcec_406b_be53_9edacc5fad53.slice/crio-26d954defda8e9fc5ddf932337327eef33fa7ec1a724592e01898b00059cca07 WatchSource:0}: Error finding container 26d954defda8e9fc5ddf932337327eef33fa7ec1a724592e01898b00059cca07: Status 404 returned error can't find the container with id 26d954defda8e9fc5ddf932337327eef33fa7ec1a724592e01898b00059cca07
Nov 02 22:08:56.071501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:56.071483    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:08:56.145776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:56.145729    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" event={"ID":"c6bf5b71-fcec-406b-be53-9edacc5fad53","Type":"ContainerStarted","Data":"26d954defda8e9fc5ddf932337327eef33fa7ec1a724592e01898b00059cca07"}
Nov 02 22:08:56.313542 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:08:56.313497    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:08:56.808343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:56.808299    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:08:56.808590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:56.808559    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy" containerID="cri-o://98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2" gracePeriod=30
Nov 02 22:08:57.155206 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:57.155168    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" event={"ID":"c6bf5b71-fcec-406b-be53-9edacc5fad53","Type":"ContainerStarted","Data":"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"}
Nov 02 22:08:57.155654 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:57.155264    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:08:57.269349 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:08:57.269294    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" podStartSLOduration=2.379285179 podStartE2EDuration="3.269276009s" totalImagesPullingTime="889.99083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:08:57.267715161 +0000 UTC m=+2035.784210957" watchObservedRunningTime="2026-11-02 22:08:57.269276009 +0000 UTC m=+2035.785771781"
Nov 02 22:09:02.112610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.112573    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:09:02.193660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193620    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.193888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193705    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.193888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193768    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.193888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193814    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193926    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.193931    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:02.194079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194000    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194053    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194288 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194085    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194288 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194135    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:02.194288 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194158    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\") pod \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\" (UID: \"a04e413b-81eb-4b44-bbba-a9fa2b17259c\") "
Nov 02 22:09:02.194288 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194277    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:02.194493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194410    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:09:02.194548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194520    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:02.195032 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.194999    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.195032 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.195030    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.195215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.195047    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.195215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.195062    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.195215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.195076    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.196401 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.196373    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:09:02.196699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.196674    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:02.196823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.196711    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:02.196823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.196791    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27" pod "a04e413b-81eb-4b44-bbba-a9fa2b17259c" (UID: "a04e413b-81eb-4b44-bbba-a9fa2b17259c"). InnerVolumeSpecName "kube-api-access-nnn27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:02.210030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.209998    5580 generic.go:356] "Generic (PLEG): container finished" podID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerID="98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2" exitCode=0
Nov 02 22:09:02.210172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.210064    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"
Nov 02 22:09:02.210172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.210077    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerDied","Data":"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"}
Nov 02 22:09:02.210172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.210134    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc" event={"ID":"a04e413b-81eb-4b44-bbba-a9fa2b17259c","Type":"ContainerDied","Data":"4469cf703895ee8b26cc34d00ec4ce289e326e9a3f0e2b09e3da1761bd40ba9a"}
Nov 02 22:09:02.210172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.210157    5580 scope.go:122] "RemoveContainer" containerID="98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"
Nov 02 22:09:02.233230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.233211    5580 scope.go:122] "RemoveContainer" containerID="64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"
Nov 02 22:09:02.243160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.243138    5580 scope.go:122] "RemoveContainer" containerID="98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"
Nov 02 22:09:02.243475 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:02.243454    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2\": container with ID starting with 98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2 not found: ID does not exist" containerID="98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"
Nov 02 22:09:02.243531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.243487    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2"} err="failed to get container status \"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2\": rpc error: code = NotFound desc = could not find container \"98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2\": container with ID starting with 98af8395bddfe71316e5554afab3f929d54607a937396b8097f99d5c918bbfc2 not found: ID does not exist"
Nov 02 22:09:02.243531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.243509    5580 scope.go:122] "RemoveContainer" containerID="64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"
Nov 02 22:09:02.243795 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:02.243778    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4\": container with ID starting with 64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4 not found: ID does not exist" containerID="64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"
Nov 02 22:09:02.243860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.243801    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4"} err="failed to get container status \"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4\": rpc error: code = NotFound desc = could not find container \"64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4\": container with ID starting with 64abb0b841120e4abfbd473c62f9ab7338c727c80e73f6fa0c205a88406f49b4 not found: ID does not exist"
Nov 02 22:09:02.297014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.296930    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.297014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.297017    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.297221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.297034    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04e413b-81eb-4b44-bbba-a9fa2b17259c-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.297221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.297049    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnn27\" (UniqueName: \"kubernetes.io/projected/a04e413b-81eb-4b44-bbba-a9fa2b17259c-kube-api-access-nnn27\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:02.440329 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.440290    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:09:02.444001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:02.443973    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3eb190-openshift-default-bdd5b9665-29mfc"]
Nov 02 22:09:03.184358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:03.184327    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:09:03.595507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:03.595416    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" path="/var/lib/kubelet/pods/a04e413b-81eb-4b44-bbba-a9fa2b17259c/volumes"
Nov 02 22:09:06.253576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.253539    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:09:06.254281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.254259    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.254415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.254289    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.254415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.254309    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.254415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.254315    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.254524 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.254461    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.263034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.263004    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.313750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.313714    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:09:06.315100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.315070    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-bdc145\""
Nov 02 22:09:06.360670 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.360557    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-bdc145-dockercfg-p6nk2\""
Nov 02 22:09:06.389985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.386022    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmhbh\" (UniqueName: \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.389985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.386120    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.500835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.500776    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.504238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.504015    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmhbh\" (UniqueName: \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.506423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.503414    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.520099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.520060    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:09:06.528205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.528171    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a04e413b-81eb-4b44-bbba-a9fa2b17259c" containerName="istio-proxy"
Nov 02 22:09:06.544881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.544841    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:09:06.546770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.544944    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.565183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.565146    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmhbh\" (UniqueName: \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\") pod \"payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.580518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.580475    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:06.607466 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.607431    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.607936 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.607901    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c578k\" (UniqueName: \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.710163 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.710109    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c578k\" (UniqueName: \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.710416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.710229    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.711273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.711243    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.767431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.767348    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c578k\" (UniqueName: \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\") pod \"payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") " pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.864077 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.864043    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:06.969074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:06.969045    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:09:06.973095 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:06.973062    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f474e96_2212_474d_b7e4_b8813cdc398e.slice/crio-aae90871460f3e25eed2c22140b1036066917c354387c3925b6be8f761395dd2 WatchSource:0}: Error finding container aae90871460f3e25eed2c22140b1036066917c354387c3925b6be8f761395dd2: Status 404 returned error can't find the container with id aae90871460f3e25eed2c22140b1036066917c354387c3925b6be8f761395dd2
Nov 02 22:09:07.247076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:07.247046    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:09:07.248198 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:07.248167    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92249a10_e668_4fcc_9bbf_8770552b2563.slice/crio-845509d5a458d9803a520c165137736481e2bcf8c8d9a2b803aa13c15068c876 WatchSource:0}: Error finding container 845509d5a458d9803a520c165137736481e2bcf8c8d9a2b803aa13c15068c876: Status 404 returned error can't find the container with id 845509d5a458d9803a520c165137736481e2bcf8c8d9a2b803aa13c15068c876
Nov 02 22:09:07.249022 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:07.248920    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" event={"ID":"1f474e96-2212-474d-b7e4-b8813cdc398e","Type":"ContainerStarted","Data":"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"}
Nov 02 22:09:07.249107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:07.249019    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" event={"ID":"1f474e96-2212-474d-b7e4-b8813cdc398e","Type":"ContainerStarted","Data":"aae90871460f3e25eed2c22140b1036066917c354387c3925b6be8f761395dd2"}
Nov 02 22:09:07.249107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:07.249055    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:07.363178 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:07.363125    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" podStartSLOduration=1.363109591 podStartE2EDuration="1.363109591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:07.360359894 +0000 UTC m=+2045.876855682" watchObservedRunningTime="2026-11-02 22:09:07.363109591 +0000 UTC m=+2045.879605364"
Nov 02 22:09:08.257470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:08.257431    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" event={"ID":"92249a10-e668-4fcc-9bbf-8770552b2563","Type":"ContainerStarted","Data":"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"}
Nov 02 22:09:08.257671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:08.257488    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" event={"ID":"92249a10-e668-4fcc-9bbf-8770552b2563","Type":"ContainerStarted","Data":"845509d5a458d9803a520c165137736481e2bcf8c8d9a2b803aa13c15068c876"}
Nov 02 22:09:08.257671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:08.257509    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:08.372897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:08.372843    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" podStartSLOduration=2.372828454 podStartE2EDuration="2.372828454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:08.369322087 +0000 UTC m=+2046.885817872" watchObservedRunningTime="2026-11-02 22:09:08.372828454 +0000 UTC m=+2046.889324251"
Nov 02 22:09:11.740252 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.740214    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:09:11.745631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.745612    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.798300 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.798268    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e2be7cc3-openshift-default-dockercfg-82vtr\""
Nov 02 22:09:11.853408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.853360    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:09:11.892657 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.892612    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.892834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.892681    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.892834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.892758    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.892834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.892824    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q49t6\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.892985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.892923    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.893044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.893032    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.893081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.893059    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.893120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.893082    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.893194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.893180    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.994792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.994656    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.994792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.994710    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.994792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.994757    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.994821    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995015    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995061    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995103    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995181    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49t6\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995233    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995241    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995586    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995622    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995776    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.995972 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.995832    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:11.999861 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:11.998895    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:12.000902 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.000880    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:12.052100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.052070    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49t6\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:12.052100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.052089    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:12.064049 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.064022    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:12.494303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.494269    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:09:12.495146 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:12.495118    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d6874f_bb2c_4142_9793_481d0b108b6b.slice/crio-4bceadb05cb741839cc33e2df22daeb2b954e2e31dcfb57ea89842f3f3ebf20e WatchSource:0}: Error finding container 4bceadb05cb741839cc33e2df22daeb2b954e2e31dcfb57ea89842f3f3ebf20e: Status 404 returned error can't find the container with id 4bceadb05cb741839cc33e2df22daeb2b954e2e31dcfb57ea89842f3f3ebf20e
Nov 02 22:09:12.498290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.498257    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:12.498380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.498329    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:12.498380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:12.498358    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:13.307169 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:13.307123    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" event={"ID":"04d6874f-bb2c-4142-9793-481d0b108b6b","Type":"ContainerStarted","Data":"b24a576683c215d918e952d23aa3404aa7b91ca92d2840a915b962303af2b96a"}
Nov 02 22:09:13.307753 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:13.307181    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" event={"ID":"04d6874f-bb2c-4142-9793-481d0b108b6b","Type":"ContainerStarted","Data":"4bceadb05cb741839cc33e2df22daeb2b954e2e31dcfb57ea89842f3f3ebf20e"}
Nov 02 22:09:13.474619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:13.474561    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podStartSLOduration=2.474540135 podStartE2EDuration="2.474540135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:13.473573526 +0000 UTC m=+2051.990069307" watchObservedRunningTime="2026-11-02 22:09:13.474540135 +0000 UTC m=+2051.991035911"
Nov 02 22:09:14.064245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:14.064209    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:14.065662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:14.065631    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 02 22:09:14.065828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:14.065684    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 02 22:09:15.065285 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:15.065228    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 02 22:09:15.065825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:15.065298    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 02 22:09:16.065428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:16.065381    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 02 22:09:16.065897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:16.065454    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 02 22:09:17.064806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:17.064770    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 02 22:09:17.065014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:17.064823    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 02 22:09:18.065277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:18.065235    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 02 22:09:18.065817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:18.065355    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 02 22:09:18.269099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:18.269062    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:09:18.860475 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:18.860389    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:09:18.860788 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:18.860746    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" podUID="5316cc5e-1937-4249-9d61-0419fbaca216" containerName="maas-api" containerID="cri-o://40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718" gracePeriod=30
Nov 02 22:09:19.137369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.136296    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:19.137369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.137302    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:19.139171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.139141    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:09:19.336977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.330896    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:09:19.343977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.338287    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:09:19.371309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.368981    5580 generic.go:356] "Generic (PLEG): container finished" podID="5316cc5e-1937-4249-9d61-0419fbaca216" containerID="40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718" exitCode=0
Nov 02 22:09:19.371309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.370258    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"
Nov 02 22:09:19.371309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.370405    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" event={"ID":"5316cc5e-1937-4249-9d61-0419fbaca216","Type":"ContainerDied","Data":"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"}
Nov 02 22:09:19.371309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.370450    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf" event={"ID":"5316cc5e-1937-4249-9d61-0419fbaca216","Type":"ContainerDied","Data":"9651c01b13f2630d247d8ddc8af68d8ad7e4b4e8b6852825c40592170218d896"}
Nov 02 22:09:19.371309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.370482    5580 scope.go:122] "RemoveContainer" containerID="40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"
Nov 02 22:09:19.415213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.414715    5580 scope.go:122] "RemoveContainer" containerID="40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"
Nov 02 22:09:19.422977 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:19.419299    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718\": container with ID starting with 40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718 not found: ID does not exist" containerID="40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"
Nov 02 22:09:19.422977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.419348    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718"} err="failed to get container status \"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718\": rpc error: code = NotFound desc = could not find container \"40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718\": container with ID starting with 40923f2f7a68144e96030da5d46befa3559dea17804ffaf6be1f8b6ff9d7b718 not found: ID does not exist"
Nov 02 22:09:19.467704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.467661    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\") pod \"5316cc5e-1937-4249-9d61-0419fbaca216\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") "
Nov 02 22:09:19.467919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.467767    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\") pod \"5316cc5e-1937-4249-9d61-0419fbaca216\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") "
Nov 02 22:09:19.467919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.467871    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") pod \"5316cc5e-1937-4249-9d61-0419fbaca216\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") "
Nov 02 22:09:19.467919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.467913    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\" (UniqueName: \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\") pod \"5316cc5e-1937-4249-9d61-0419fbaca216\" (UID: \"5316cc5e-1937-4249-9d61-0419fbaca216\") "
Nov 02 22:09:19.468280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.468252    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca" pod "5316cc5e-1937-4249-9d61-0419fbaca216" (UID: "5316cc5e-1937-4249-9d61-0419fbaca216"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:09:19.469572 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.469546    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5316cc5e-1937-4249-9d61-0419fbaca216-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:19.474382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.471848    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs" pod "5316cc5e-1937-4249-9d61-0419fbaca216" (UID: "5316cc5e-1937-4249-9d61-0419fbaca216"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:09:19.476675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.476636    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm" pod "5316cc5e-1937-4249-9d61-0419fbaca216" (UID: "5316cc5e-1937-4249-9d61-0419fbaca216"). InnerVolumeSpecName "kube-api-access-wpkfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:19.479159 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.479126    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls" pod "5316cc5e-1937-4249-9d61-0419fbaca216" (UID: "5316cc5e-1937-4249-9d61-0419fbaca216"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:09:19.571211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.571134    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:19.571211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.571170    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5316cc5e-1937-4249-9d61-0419fbaca216-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:19.571211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.571185    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpkfm\" (UniqueName: \"kubernetes.io/projected/5316cc5e-1937-4249-9d61-0419fbaca216-kube-api-access-wpkfm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:19.785426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.785378    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:09:19.789564 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:19.789529    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-224f53-6b88dd664c-fjxmf"]
Nov 02 22:09:21.596920 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:21.596885    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5316cc5e-1937-4249-9d61-0419fbaca216" path="/var/lib/kubelet/pods/5316cc5e-1937-4249-9d61-0419fbaca216/volumes"
Nov 02 22:09:25.941019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:25.939487    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:09:26.313688 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:26.313600    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:09:26.863234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.863199    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:09:26.864010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.863992    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5316cc5e-1937-4249-9d61-0419fbaca216" containerName="maas-api"
Nov 02 22:09:26.864061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.864011    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5316cc5e-1937-4249-9d61-0419fbaca216" containerName="maas-api"
Nov 02 22:09:26.864148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.864139    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5316cc5e-1937-4249-9d61-0419fbaca216" containerName="maas-api"
Nov 02 22:09:26.867785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.867760    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:26.912502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.912457    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:09:26.917521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.917492    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:26.967205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:26.967166    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:09:27.007966 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.007898    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.008215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.007978    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.008215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008193    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008240    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008385    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008493    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008608    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008638    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008658    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008676    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrnp8\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008702    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008731    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008762    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008791    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008858    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008901    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2z5d\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.008930    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.009257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.009001    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.019090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.018417    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:09:27.110599 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110562    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110643    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110674    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110707    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110734    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrnp8\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110758    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.110807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110790    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110833    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110867    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.110950    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111055    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2z5d\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111095    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111109    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.111218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111140    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.111576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111223    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111258    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111408    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111753 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111636    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.111890 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.111867    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.112163 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.112137    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.112436 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.112412    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.112757 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.112676    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.112931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.112911    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.113092 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.113076    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.113234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.113217    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.114207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.114146    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.114207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.113372    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.114207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.114008    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.114364 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.113110    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.115240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.115216    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.115445 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.115428    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.116625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.116593    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.167214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.167183    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.167380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.167272    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.167380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.167323    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrnp8\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.167670 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.167647    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2z5d\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\") pod \"e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") " pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.184497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.184456    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:27.231730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.231687    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:27.624548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.624510    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:09:27.664505 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:27.664470    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8dc48b_3c3a_4def_abf8_1638d8e47a4e.slice/crio-608245cfd1230a4c7913b7065fbf2042da18cc6c943bd3b6100afb5e37c51a06 WatchSource:0}: Error finding container 608245cfd1230a4c7913b7065fbf2042da18cc6c943bd3b6100afb5e37c51a06: Status 404 returned error can't find the container with id 608245cfd1230a4c7913b7065fbf2042da18cc6c943bd3b6100afb5e37c51a06
Nov 02 22:09:27.667640 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.667603    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:27.667754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.667678    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:27.667754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.667709    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:27.680001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.679950    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:09:27.682470 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:27.682436    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a3988e_fac7_4132_93b5_6f97f7ac7c32.slice/crio-72d661f8096cc4644266f3f55fa920a971cf944ff7ef0b76329e0d6e41f8b106 WatchSource:0}: Error finding container 72d661f8096cc4644266f3f55fa920a971cf944ff7ef0b76329e0d6e41f8b106: Status 404 returned error can't find the container with id 72d661f8096cc4644266f3f55fa920a971cf944ff7ef0b76329e0d6e41f8b106
Nov 02 22:09:27.690198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.690041    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:27.690198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.690148    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:27.690198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:27.690198    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:28.237065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.221030    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:09:28.237065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.221438    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy" containerID="cri-o://d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27" gracePeriod=30
Nov 02 22:09:28.481883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.481782    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" event={"ID":"b9a3988e-fac7-4132-93b5-6f97f7ac7c32","Type":"ContainerStarted","Data":"056012b158459e17a380107beed2497fdaeccb94980825ad067ef2fe2d63dd78"}
Nov 02 22:09:28.482217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.482188    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" event={"ID":"b9a3988e-fac7-4132-93b5-6f97f7ac7c32","Type":"ContainerStarted","Data":"72d661f8096cc4644266f3f55fa920a971cf944ff7ef0b76329e0d6e41f8b106"}
Nov 02 22:09:28.486216 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.486176    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" event={"ID":"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e","Type":"ContainerStarted","Data":"00c13ed93046e9de34e53db6349345543e54778fea7bb6e8501b1e10723a5ca3"}
Nov 02 22:09:28.486484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.486450    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" event={"ID":"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e","Type":"ContainerStarted","Data":"608245cfd1230a4c7913b7065fbf2042da18cc6c943bd3b6100afb5e37c51a06"}
Nov 02 22:09:28.772026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.771202    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podStartSLOduration=2.771179519 podStartE2EDuration="2.771179519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:28.766817279 +0000 UTC m=+2067.283313053" watchObservedRunningTime="2026-11-02 22:09:28.771179519 +0000 UTC m=+2067.287675293"
Nov 02 22:09:28.772193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:28.772100    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podStartSLOduration=2.772089094 podStartE2EDuration="2.772089094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:28.65245959 +0000 UTC m=+2067.168955364" watchObservedRunningTime="2026-11-02 22:09:28.772089094 +0000 UTC m=+2067.288584868"
Nov 02 22:09:29.196982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.193435    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:29.196982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.193594    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:29.196982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.193642    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:29.246067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.234136    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:29.246067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.236714    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:29.246067 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:29.236797    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:30.188994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.185384    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:30.188994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.185462    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:30.234469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.232530    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:30.234469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.232595    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:30.359992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.356471    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:09:30.367215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.366512    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.522044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.519976    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-027bdf-openshift-default-dockercfg-ngrbg\""
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.856506    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.858120    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.858217    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.858306    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.858363    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.858402    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.859169    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.859219    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.859312    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jntst\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.867058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.859357    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.966989    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntst\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967060    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967111    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967220    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967323    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967389    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967436    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967535    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.967586    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.968131    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.988996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.976159    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.988996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.976450    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.988996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.976682    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.988996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.977305    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:30.988996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:30.984775    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:31.063982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.051572    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:31.065949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.064203    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:31.089979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.081125    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jntst\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\") pod \"e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") " pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:31.171901 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.168643    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:31.205994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.205489    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:31.205994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.205566    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:31.251561 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.251360    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:31.251968 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:31.251528    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:32.108002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.077997    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:09:32.164416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.161642    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:32.164416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.161824    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:32.164416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.161929    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:32.223983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.207583    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:32.223983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.207648    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:32.261034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.259063    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:32.261034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:32.259144    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:33.187981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.187514    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:33.187981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.187599    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:33.237982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.234146    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:33.237982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.234211    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:33.281551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.269496    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerStarted","Data":"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"}
Nov 02 22:09:33.281551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.269570    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerStarted","Data":"036b12871f27fc2603e013149f095bb168f1689c68d8919c262c091b8dcd8232"}
Nov 02 22:09:33.459045 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:33.447731    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podStartSLOduration=3.447705851 podStartE2EDuration="3.447705851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:33.444746902 +0000 UTC m=+2071.961242677" watchObservedRunningTime="2026-11-02 22:09:33.447705851 +0000 UTC m=+2071.964201626"
Nov 02 22:09:34.175028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.174994    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:34.233215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.226113    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:34.233215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.226202    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:34.243683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.243351    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:34.244371 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.244241    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:34.244742 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.244582    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:34.273988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.273928    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:09:34.356984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.356656    5580 generic.go:356] "Generic (PLEG): container finished" podID="d26ff062-e4ec-4516-9691-799cf88e4626" containerID="d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27" exitCode=0
Nov 02 22:09:34.359023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.357353    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"
Nov 02 22:09:34.359023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.358051    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerDied","Data":"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"}
Nov 02 22:09:34.359023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.358136    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76" event={"ID":"d26ff062-e4ec-4516-9691-799cf88e4626","Type":"ContainerDied","Data":"73680907fa4b42fcf1f249fe478e1f8059e6ba37c473a39828cd7183dece89cc"}
Nov 02 22:09:34.359023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.358186    5580 scope.go:122] "RemoveContainer" containerID="d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"
Nov 02 22:09:34.359023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.358351    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:34.361812 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.361779    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.406620    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.406714    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.406757    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.406804    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.407058    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.407136    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.407197    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.407331    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.407368    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\") pod \"d26ff062-e4ec-4516-9691-799cf88e4626\" (UID: \"d26ff062-e4ec-4516-9691-799cf88e4626\") "
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.413187    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.414408    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:34.426198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.423077    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:34.435018 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.434982    5580 scope.go:122] "RemoveContainer" containerID="91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"
Nov 02 22:09:34.438931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.438674    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:34.438931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.438851    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:34.443307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.443283    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:34.443596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.443539    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:09:34.445467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.445417    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:34.447721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.447685    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt" pod "d26ff062-e4ec-4516-9691-799cf88e4626" (UID: "d26ff062-e4ec-4516-9691-799cf88e4626"). InnerVolumeSpecName "kube-api-access-zn7dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:34.511555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.511510    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-kube-api-access-zn7dt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.531584 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.531549    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.531719 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.531706    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d26ff062-e4ec-4516-9691-799cf88e4626-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.531810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.531799    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d26ff062-e4ec-4516-9691-799cf88e4626-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.531890 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.531879    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.532011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.531997    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.532113 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.532101    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.532202 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.532192    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d26ff062-e4ec-4516-9691-799cf88e4626-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.532281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.532270    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d26ff062-e4ec-4516-9691-799cf88e4626-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.534554    5580 scope.go:122] "RemoveContainer" containerID="d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:34.535684    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27\": container with ID starting with d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27 not found: ID does not exist" containerID="d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.535744    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27"} err="failed to get container status \"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27\": rpc error: code = NotFound desc = could not find container \"d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27\": container with ID starting with d833c8c7eb1358ad2a8eb96fe2caf90a07981d0fa6ff2b6a71526ececb608b27 not found: ID does not exist"
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.535792    5580 scope.go:122] "RemoveContainer" containerID="91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:34.537257    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66\": container with ID starting with 91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66 not found: ID does not exist" containerID="91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"
Nov 02 22:09:34.537473 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.537286    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66"} err="failed to get container status \"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66\": rpc error: code = NotFound desc = could not find container \"91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66\": container with ID starting with 91c55d34a2f135f36f98c1fc362376a4f1703f41b9930f1042af63e0e2b0bf66 not found: ID does not exist"
Nov 02 22:09:34.860110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.860022    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:09:34.863667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:34.863641    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-224f53-openshift-default-6599f64fdf-kfr76"]
Nov 02 22:09:35.185499 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:35.185443    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Nov 02 22:09:35.185698 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:35.185507    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Nov 02 22:09:35.232596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:35.232552    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 02 22:09:35.232831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:35.232640    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 02 22:09:35.604461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:35.604357    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" path="/var/lib/kubelet/pods/d26ff062-e4ec-4516-9691-799cf88e4626/volumes"
Nov 02 22:09:36.191001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.190948    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:36.191488 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.191465    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:36.192199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.192178    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:09:36.238238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.238206    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:36.238642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.238616    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:36.239432 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:36.239410    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:09:39.422362 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.422329    5580 generic.go:356] "Generic (PLEG): container finished" podID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerID="498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a" exitCode=0
Nov 02 22:09:39.423034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.422406    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerDied","Data":"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"}
Nov 02 22:09:39.423123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.423101    5580 scope.go:122] "RemoveContainer" containerID="498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"
Nov 02 22:09:39.425184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.425152    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:39.425293 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.425229    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:39.425293 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:39.425263    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:09:40.414556 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.414521    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:09:40.415325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.415301    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.415325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.415320    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.415552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.415366    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.415552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.415373    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.415552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.415493    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.422687 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.422658    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.471991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.461033    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerStarted","Data":"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"}
Nov 02 22:09:40.472626 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.472596    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:09:40.479987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.474362    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.479987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.474555    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd2pd\" (UniqueName: \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.553539 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.524316    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-027bdf-dockercfg-5bhnw\""
Nov 02 22:09:40.553789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.545206    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-027bdf\""
Nov 02 22:09:40.585404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.576585    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd2pd\" (UniqueName: \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.585404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.582142    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.585404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.583355    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.639993 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.638480    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:09:40.643025 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.641478    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d26ff062-e4ec-4516-9691-799cf88e4626" containerName="istio-proxy"
Nov 02 22:09:40.682584 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.682547    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd2pd\" (UniqueName: \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\") pod \"payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.718982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.707600    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:09:40.718982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.707741    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.803525 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.803126    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.803525 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.803346    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvzxf\" (UniqueName: \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.868194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.868155    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:40.905629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.904581    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvzxf\" (UniqueName: \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.905629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.904731    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.905785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.905726    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:40.964793 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:40.964741    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvzxf\" (UniqueName: \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\") pod \"payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:41.052301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.052266    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:41.307354 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:41.307317    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d213f6c_44ab_454a_adcd_90e7dea9a92e.slice/crio-8f944e421613a9a1504cb26b56423fc7329f390817e65741ae9e0d113f1aa87c WatchSource:0}: Error finding container 8f944e421613a9a1504cb26b56423fc7329f390817e65741ae9e0d113f1aa87c: Status 404 returned error can't find the container with id 8f944e421613a9a1504cb26b56423fc7329f390817e65741ae9e0d113f1aa87c
Nov 02 22:09:41.311200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.309085    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:09:41.320755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.320726    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:09:41.322506 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.322473    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 02 22:09:41.322637 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.322533    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 02 22:09:41.470659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.470618    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" event={"ID":"3d213f6c-44ab-454a-adcd-90e7dea9a92e","Type":"ContainerStarted","Data":"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"}
Nov 02 22:09:41.471231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.470682    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" event={"ID":"3d213f6c-44ab-454a-adcd-90e7dea9a92e","Type":"ContainerStarted","Data":"8f944e421613a9a1504cb26b56423fc7329f390817e65741ae9e0d113f1aa87c"}
Nov 02 22:09:41.471231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.470841    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:41.506158 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:41.506108    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:09:41.511821 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:41.511788    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86701424_0da1_4aed_8f2d_cbe63230198e.slice/crio-423e6a00c85ded7880743a40b4cddd814ccaec2c20c80b321c1dc791be9c8b67 WatchSource:0}: Error finding container 423e6a00c85ded7880743a40b4cddd814ccaec2c20c80b321c1dc791be9c8b67: Status 404 returned error can't find the container with id 423e6a00c85ded7880743a40b4cddd814ccaec2c20c80b321c1dc791be9c8b67
Nov 02 22:09:42.320895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.320789    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 02 22:09:42.320895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.320865    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 02 22:09:42.503039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.502995    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" event={"ID":"86701424-0da1-4aed-8f2d-cbe63230198e","Type":"ContainerStarted","Data":"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"}
Nov 02 22:09:42.503538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.503064    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" event={"ID":"86701424-0da1-4aed-8f2d-cbe63230198e","Type":"ContainerStarted","Data":"423e6a00c85ded7880743a40b4cddd814ccaec2c20c80b321c1dc791be9c8b67"}
Nov 02 22:09:42.503538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.503284    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:42.614874 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.614759    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" podStartSLOduration=2.614741606 podStartE2EDuration="2.614741606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:42.614225519 +0000 UTC m=+2081.130721313" watchObservedRunningTime="2026-11-02 22:09:42.614741606 +0000 UTC m=+2081.131237380"
Nov 02 22:09:42.617387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:42.617349    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" podStartSLOduration=2.61733859 podStartE2EDuration="2.61733859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:41.606596997 +0000 UTC m=+2080.123092771" watchObservedRunningTime="2026-11-02 22:09:42.61733859 +0000 UTC m=+2081.133834406"
Nov 02 22:09:43.333983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:43.324213    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 02 22:09:43.333983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:43.324280    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 02 22:09:43.762296 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:43.762253    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:09:44.320699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:44.320655    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 02 22:09:44.320899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:44.320718    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 02 22:09:45.320934 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:45.320887    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 02 22:09:45.321453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:45.320972    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 02 22:09:46.325811 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:46.325776    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:09:46.327237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:46.327209    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:09:52.506773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:52.506738    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:09:53.513788 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:53.513758    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:09:56.040765 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.040723    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:09:56.045522 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.045499    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.079888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.079845    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9j7c\" (UniqueName: \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.080130 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.080035    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.080269 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.080247    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.080322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.080297    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.100661 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.100622    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:09:56.145545 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.145501    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-17435d-serving-cert\""
Nov 02 22:09:56.182612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.182566    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9j7c\" (UniqueName: \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.182823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.182725    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.182970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.182935    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.183056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.183015    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.183269 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:56.183245    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-serving-cert: secret "maas-api-e2e-inf-a-17435d-serving-cert" not found
Nov 02 22:09:56.183465 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:56.183447    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls podName:cdf11c5b-fc58-4cdb-a515-f93fbeb4735c nodeName:}" failed. No retries permitted until 2026-11-02 22:09:56.683424275 +0000 UTC m=+2095.199920043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls") pod "maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" (UID: "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c") : secret "maas-api-e2e-inf-a-17435d-serving-cert" not found
Nov 02 22:09:56.183806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.183768    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.185726 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.185697    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.239260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.239225    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9j7c\" (UniqueName: \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.332503 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:09:56.332287    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:09:56.693758 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.693720    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.696204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.696173    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") pod \"maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:56.961907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:56.961817    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:57.345745 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:57.345708    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:09:57.346599 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:09:57.346573    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf11c5b_fc58_4cdb_a515_f93fbeb4735c.slice/crio-91a8b0a70a04c7480f7c40bcf2532694cb133abd9a38e53a6dc159c8f28d837e WatchSource:0}: Error finding container 91a8b0a70a04c7480f7c40bcf2532694cb133abd9a38e53a6dc159c8f28d837e: Status 404 returned error can't find the container with id 91a8b0a70a04c7480f7c40bcf2532694cb133abd9a38e53a6dc159c8f28d837e
Nov 02 22:09:57.617539 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:57.617447    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" event={"ID":"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c","Type":"ContainerStarted","Data":"91a8b0a70a04c7480f7c40bcf2532694cb133abd9a38e53a6dc159c8f28d837e"}
Nov 02 22:09:58.626776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:58.626664    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" event={"ID":"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c","Type":"ContainerStarted","Data":"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"}
Nov 02 22:09:58.626776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:58.626766    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:09:58.765122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:09:58.765064    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" podStartSLOduration=2.904514099 podStartE2EDuration="3.765046214s" totalImagesPullingTime="860.532115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:58.763481467 +0000 UTC m=+2097.279977240" watchObservedRunningTime="2026-11-02 22:09:58.765046214 +0000 UTC m=+2097.281541988"
Nov 02 22:10:02.137527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:02.137497    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:10:02.143985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:02.143948    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:10:02.149157 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:02.149134    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:10:02.155434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:02.155406    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:10:04.639591 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:04.639560    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:10:15.736204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:15.736171    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:10:15.842741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:15.842682    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:10:15.846828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:15.846794    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 22:10:15.847082 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:15.847057    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy" containerID="cri-o://cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c" gracePeriod=30
Nov 02 22:10:16.287982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:16.283436    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:10:16.325001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:16.322102    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:10:20.737453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:20.737373    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerName="payload-processing" containerID="cri-o://214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122" gracePeriod=25
Nov 02 22:10:20.843843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:20.843770    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" podUID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" containerName="payload-pre-processing" containerID="cri-o://0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999" gracePeriod=25
Nov 02 22:10:21.083202 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.083165    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:10:21.155839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.152497    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.227532    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.231692    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\") pod \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") "
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.232275    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume" pod "24efedd7-9d0c-47b7-a6dc-7abf2347434b" (UID: "24efedd7-9d0c-47b7-a6dc-7abf2347434b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.232570    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\" (UniqueName: \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\") pod \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\" (UID: \"24efedd7-9d0c-47b7-a6dc-7abf2347434b\") "
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.232663    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\" (UniqueName: \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\") pod \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") "
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.233426    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\") pod \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\" (UID: \"b1ba0ad0-fee6-48a0-a360-75228b16c54d\") "
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.234245    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24efedd7-9d0c-47b7-a6dc-7abf2347434b-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.234729    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume" pod "b1ba0ad0-fee6-48a0-a360-75228b16c54d" (UID: "b1ba0ad0-fee6-48a0-a360-75228b16c54d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:21.296179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.291286    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" podUID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" containerName="payload-processing" containerID="cri-o://c546a7d84c55507f8b836ffd7e5ec4d08ce7ec8da45847fee15732cccb4d6d9a" gracePeriod=25
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.338878    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.338946    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339080    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339188    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339286    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339338    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339449    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339482    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.345979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.339590    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\") pod \"c66118af-5aaf-4729-b97f-ac8b5e424f31\" (UID: \"c66118af-5aaf-4729-b97f-ac8b5e424f31\") "
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.351544    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.352430    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c66118af-5aaf-4729-b97f-ac8b5e424f31-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.352459    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ba0ad0-fee6-48a0-a360-75228b16c54d-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.352781    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.353285    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.353525    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.354135    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:21.361190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.354369    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" podUID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" containerName="payload-pre-processing" containerID="cri-o://50b5639bafb974a233495cf03d1471a633f6b815eb19daf52c6e11c7d5fb6334" gracePeriod=25
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.536376    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.538077    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541409    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541442    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541456    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541474    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541489    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.548104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.541502    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.565880 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.565716    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m" pod "b1ba0ad0-fee6-48a0-a360-75228b16c54d" (UID: "b1ba0ad0-fee6-48a0-a360-75228b16c54d"). InnerVolumeSpecName "kube-api-access-djw4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:21.567531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.567479    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "kube-api-access-4c7hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:21.575559 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.575332    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy" pod "c66118af-5aaf-4729-b97f-ac8b5e424f31" (UID: "c66118af-5aaf-4729-b97f-ac8b5e424f31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:21.578010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.577155    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f" pod "24efedd7-9d0c-47b7-a6dc-7abf2347434b" (UID: "24efedd7-9d0c-47b7-a6dc-7abf2347434b"). InnerVolumeSpecName "kube-api-access-8624f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:21.645025 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.644992    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c66118af-5aaf-4729-b97f-ac8b5e424f31-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.645275 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.645254    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c7hq\" (UniqueName: \"kubernetes.io/projected/c66118af-5aaf-4729-b97f-ac8b5e424f31-kube-api-access-4c7hq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.645380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.645367    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8624f\" (UniqueName: \"kubernetes.io/projected/24efedd7-9d0c-47b7-a6dc-7abf2347434b-kube-api-access-8624f\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.645461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.645448    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djw4m\" (UniqueName: \"kubernetes.io/projected/b1ba0ad0-fee6-48a0-a360-75228b16c54d-kube-api-access-djw4m\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:21.652899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.652861    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:10:21.833714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.833173    5580 generic.go:356] "Generic (PLEG): container finished" podID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" containerID="c546a7d84c55507f8b836ffd7e5ec4d08ce7ec8da45847fee15732cccb4d6d9a" exitCode=0
Nov 02 22:10:21.833714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.833349    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" event={"ID":"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d","Type":"ContainerDied","Data":"c546a7d84c55507f8b836ffd7e5ec4d08ce7ec8da45847fee15732cccb4d6d9a"}
Nov 02 22:10:21.837297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.835415    5580 generic.go:356] "Generic (PLEG): container finished" podID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerID="cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c" exitCode=0
Nov 02 22:10:21.837297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.835628    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"
Nov 02 22:10:21.837297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.836496    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" event={"ID":"c66118af-5aaf-4729-b97f-ac8b5e424f31","Type":"ContainerDied","Data":"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"}
Nov 02 22:10:21.837297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.836553    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l" event={"ID":"c66118af-5aaf-4729-b97f-ac8b5e424f31","Type":"ContainerDied","Data":"0100d660f0b2e0f915344eea97e857c43848ec5d8d3ceefae3a17f00432f1813"}
Nov 02 22:10:21.837297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.836585    5580 scope.go:122] "RemoveContainer" containerID="cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"
Nov 02 22:10:21.840302 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.840173    5580 generic.go:356] "Generic (PLEG): container finished" podID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerID="214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122" exitCode=0
Nov 02 22:10:21.840302 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.840277    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"
Nov 02 22:10:21.840580 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.840274    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" event={"ID":"b1ba0ad0-fee6-48a0-a360-75228b16c54d","Type":"ContainerDied","Data":"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"}
Nov 02 22:10:21.840580 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.840544    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" event={"ID":"b1ba0ad0-fee6-48a0-a360-75228b16c54d","Type":"ContainerDied","Data":"567fb26b66943b0b9f3519de6077b452dac06fa2e3e76751985cda0d1593b502"}
Nov 02 22:10:21.845062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.844043    5580 generic.go:356] "Generic (PLEG): container finished" podID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" containerID="0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999" exitCode=0
Nov 02 22:10:21.845062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.844179    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" event={"ID":"24efedd7-9d0c-47b7-a6dc-7abf2347434b","Type":"ContainerDied","Data":"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"}
Nov 02 22:10:21.845062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.844218    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp" event={"ID":"24efedd7-9d0c-47b7-a6dc-7abf2347434b","Type":"ContainerDied","Data":"01ccc825670f1b7cd0485672636ae47f3121684d3687e5ffa507062613dc6ba8"}
Nov 02 22:10:21.845062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.844296    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"
Nov 02 22:10:21.850519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.850393    5580 generic.go:356] "Generic (PLEG): container finished" podID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" containerID="50b5639bafb974a233495cf03d1471a633f6b815eb19daf52c6e11c7d5fb6334" exitCode=0
Nov 02 22:10:21.850519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.850473    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" event={"ID":"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758","Type":"ContainerDied","Data":"50b5639bafb974a233495cf03d1471a633f6b815eb19daf52c6e11c7d5fb6334"}
Nov 02 22:10:21.859349 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.859231    5580 scope.go:122] "RemoveContainer" containerID="cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"
Nov 02 22:10:21.860369 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:21.859833    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c\": container with ID starting with cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c not found: ID does not exist" containerID="cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"
Nov 02 22:10:21.860369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.859872    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c"} err="failed to get container status \"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c\": rpc error: code = NotFound desc = could not find container \"cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c\": container with ID starting with cc31a0b4d2a00e0f04007f02cd423ad58bd86ff92b5fdf9a89df71399c62722c not found: ID does not exist"
Nov 02 22:10:21.860369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.859897    5580 scope.go:122] "RemoveContainer" containerID="214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"
Nov 02 22:10:21.878444 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.878409    5580 scope.go:122] "RemoveContainer" containerID="214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"
Nov 02 22:10:21.878917 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:21.878881    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122\": container with ID starting with 214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122 not found: ID does not exist" containerID="214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"
Nov 02 22:10:21.879136 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.878930    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122"} err="failed to get container status \"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122\": rpc error: code = NotFound desc = could not find container \"214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122\": container with ID starting with 214fcbfb0b3ab4945d4da5ad575c1b1cc068fb514f85c035a0daa0498006d122 not found: ID does not exist"
Nov 02 22:10:21.879136 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.878975    5580 scope.go:122] "RemoveContainer" containerID="0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"
Nov 02 22:10:21.891069 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.891041    5580 scope.go:122] "RemoveContainer" containerID="0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"
Nov 02 22:10:21.891428 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:21.891402    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999\": container with ID starting with 0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999 not found: ID does not exist" containerID="0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"
Nov 02 22:10:21.891510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.891439    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999"} err="failed to get container status \"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999\": rpc error: code = NotFound desc = could not find container \"0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999\": container with ID starting with 0b2e2ec1f97c432efe36c31135e1618d8d4069dc244fc58b2e79f37359d1c999 not found: ID does not exist"
Nov 02 22:10:21.951061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.951026    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:10:21.953763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:21.953739    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:10:22.002570 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.002530    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:10:22.006072 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.006039    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz"]
Nov 02 22:10:22.025413 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.025371    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-588b5c-7b6965965c-dqpjz" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.114:50052\" within 1s: context deadline exceeded"
Nov 02 22:10:22.025552 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:10:22.025510    5580 logging.go:55] [core] [Channel #1451 SubChannel #1452]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.114:50052", ServerName: "10.134.0.114:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.114:50052: operation was canceled"
Nov 02 22:10:22.055822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.055780    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\" (UniqueName: \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\") pod \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") "
Nov 02 22:10:22.056049 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.055888    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\") pod \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\" (UID: \"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d\") "
Nov 02 22:10:22.056121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056075    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\" (UniqueName: \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\") pod \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") "
Nov 02 22:10:22.056214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056197    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\") pod \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\" (UID: \"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758\") "
Nov 02 22:10:22.056316 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056286    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume" pod "771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" (UID: "771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:22.056574 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056545    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume" pod "4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" (UID: "4f7e38bd-99b1-4b0e-a037-0fc95bc4a758"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:22.056891 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056872    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:22.056952 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.056897    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:22.058334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.058308    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq" pod "4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" (UID: "4f7e38bd-99b1-4b0e-a037-0fc95bc4a758"). InnerVolumeSpecName "kube-api-access-bqhlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:22.058435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.058335    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t" pod "771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" (UID: "771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d"). InnerVolumeSpecName "kube-api-access-vk25t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:22.114483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.114369    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:10:22.118797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.118769    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-588b5c-759cfbddc-bflcp"]
Nov 02 22:10:22.158665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.158634    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqhlq\" (UniqueName: \"kubernetes.io/projected/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758-kube-api-access-bqhlq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:22.158665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.158662    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk25t\" (UniqueName: \"kubernetes.io/projected/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d-kube-api-access-vk25t\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:22.226268 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.226231    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 22:10:22.232405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.232376    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-5dq7l"]
Nov 02 22:10:22.859496 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.859465    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"
Nov 02 22:10:22.860042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.859499    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5" event={"ID":"771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d","Type":"ContainerDied","Data":"d443802a426684baeb299401d113cfd8a286883505df73c846d37c7899bdbcf6"}
Nov 02 22:10:22.860042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.859573    5580 scope.go:122] "RemoveContainer" containerID="c546a7d84c55507f8b836ffd7e5ec4d08ce7ec8da45847fee15732cccb4d6d9a"
Nov 02 22:10:22.865412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.865384    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb" event={"ID":"4f7e38bd-99b1-4b0e-a037-0fc95bc4a758","Type":"ContainerDied","Data":"b31109be5a3ead69d99bb655e5393b8a71a4fabad9f659048e1070aafb0f464d"}
Nov 02 22:10:22.865563 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.865414    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"
Nov 02 22:10:22.873041 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:22.873020    5580 scope.go:122] "RemoveContainer" containerID="50b5639bafb974a233495cf03d1471a633f6b815eb19daf52c6e11c7d5fb6334"
Nov 02 22:10:23.039042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.039001    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:10:23.045167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.045112    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2be7cc3-7b54668dbc-d4zl5"]
Nov 02 22:10:23.156038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.155724    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:10:23.162019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.161925    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2be7cc3-598d7c6487-nx2sb"]
Nov 02 22:10:23.669981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.666491    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" path="/var/lib/kubelet/pods/24efedd7-9d0c-47b7-a6dc-7abf2347434b/volumes"
Nov 02 22:10:23.682486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.672477    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" path="/var/lib/kubelet/pods/4f7e38bd-99b1-4b0e-a037-0fc95bc4a758/volumes"
Nov 02 22:10:23.682486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.673304    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" path="/var/lib/kubelet/pods/771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d/volumes"
Nov 02 22:10:23.682486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.673947    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" path="/var/lib/kubelet/pods/b1ba0ad0-fee6-48a0-a360-75228b16c54d/volumes"
Nov 02 22:10:23.682486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:23.680159    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" path="/var/lib/kubelet/pods/c66118af-5aaf-4729-b97f-ac8b5e424f31/volumes"
Nov 02 22:10:26.312698 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:26.312663    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:10:28.044817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:28.044778    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:10:28.045291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:28.045080    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" containerID="cri-o://9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7" gracePeriod=30
Nov 02 22:10:30.568527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.568490    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:10:30.569281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569260    5580 cpu_manager.go:414] "containerMap: removing container" podUID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" containerName="payload-processing"
Nov 02 22:10:30.569342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569282    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" containerName="payload-processing"
Nov 02 22:10:30.569342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569300    5580 cpu_manager.go:414] "containerMap: removing container" podUID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" containerName="payload-pre-processing"
Nov 02 22:10:30.569342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569306    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" containerName="payload-pre-processing"
Nov 02 22:10:30.569342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569321    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerName="payload-processing"
Nov 02 22:10:30.569342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569326    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerName="payload-processing"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569350    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" containerName="payload-pre-processing"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569356    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" containerName="payload-pre-processing"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569368    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569372    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569482    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f7e38bd-99b1-4b0e-a037-0fc95bc4a758" containerName="payload-pre-processing"
Nov 02 22:10:30.569501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569496    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c66118af-5aaf-4729-b97f-ac8b5e424f31" containerName="istio-proxy"
Nov 02 22:10:30.569782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569509    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="771f085d-b1f8-4afd-b4ba-e5c4f60dcf9d" containerName="payload-processing"
Nov 02 22:10:30.569782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569518    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="24efedd7-9d0c-47b7-a6dc-7abf2347434b" containerName="payload-pre-processing"
Nov 02 22:10:30.569782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.569527    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1ba0ad0-fee6-48a0-a360-75228b16c54d" containerName="payload-processing"
Nov 02 22:10:30.573649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.573624    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.627120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.626350    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1789c6-openshift-default-dockercfg-lk7b7\""
Nov 02 22:10:30.703100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.703054    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:10:30.729073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.729027    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.729371 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.729346    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdprd\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.729570 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.729550    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.729710 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.729696    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.729900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.729879    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.730185 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.730164    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.733989 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.730405    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.734339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.734314    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.743128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.743081    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853513    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853624    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853671    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853726    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853769    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdprd\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.853883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853857    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.854115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853907    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.854115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.853983    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.854115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.854102    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.854677 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.854651    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.855613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.855587    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.856015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.855978    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.856452 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.856431    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.858474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.858359    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.863096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.861691    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.863838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.863794    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.914950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.914909    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:30.915243 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:30.915093    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdprd\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\") pod \"e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") " pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:31.190526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.190451    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:31.329123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.329084    5580 patch_prober.go:29] interesting pod/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:10:31.329327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.329163    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:10:31.626841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.626761    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:10:31.667389 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:10:31.667356    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d29e7f_e347_453d_8e46_abdc72da06b2.slice/crio-cb35e74ca024caf8d4b1e67d994cd3bc4a691b0aa9387147494c075b5f156cb6 WatchSource:0}: Error finding container cb35e74ca024caf8d4b1e67d994cd3bc4a691b0aa9387147494c075b5f156cb6: Status 404 returned error can't find the container with id cb35e74ca024caf8d4b1e67d994cd3bc4a691b0aa9387147494c075b5f156cb6
Nov 02 22:10:31.670637 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.670595    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:10:31.670750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.670682    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:10:31.670750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.670725    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:10:31.946943 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.946902    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerStarted","Data":"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"}
Nov 02 22:10:31.947141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:31.946978    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerStarted","Data":"cb35e74ca024caf8d4b1e67d994cd3bc4a691b0aa9387147494c075b5f156cb6"}
Nov 02 22:10:32.072981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:32.067533    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podStartSLOduration=2.067510642 podStartE2EDuration="2.067510642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:32.065440596 +0000 UTC m=+2130.581936370" watchObservedRunningTime="2026-11-02 22:10:32.067510642 +0000 UTC m=+2130.584006417"
Nov 02 22:10:32.193978 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:32.191613    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:32.201137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:32.195421    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:10:32.201137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:32.195508    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:10:33.196427 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.196387    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:33.361076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.351499    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410537    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410592    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410737    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410774    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410806    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410878    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.410923    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.427053    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.428005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.427266    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\") pod \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\" (UID: \"3d9959ae-f7d0-48c3-8370-3647e4ca50e1\") "
Nov 02 22:10:33.441013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.438207    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:33.441013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.438852    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:33.451014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.448614    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:33.451014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.448800    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:33.451439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.451400    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:33.453398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.453368    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:33.454743 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.454705    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:10:33.454916 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.454762    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "kube-api-access-wrf72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:33.464029 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.461676    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy" pod "3d9959ae-f7d0-48c3-8370-3647e4ca50e1" (UID: "3d9959ae-f7d0-48c3-8370-3647e4ca50e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:10:33.533352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533312    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533351    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533367    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533382    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533395    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533409    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533422    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533436    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.533635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.533449    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrf72\" (UniqueName: \"kubernetes.io/projected/3d9959ae-f7d0-48c3-8370-3647e4ca50e1-kube-api-access-wrf72\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:33.963643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.963606    5580 generic.go:356] "Generic (PLEG): container finished" podID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerID="9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7" exitCode=0
Nov 02 22:10:33.963869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.963686    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"
Nov 02 22:10:33.963869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.963730    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerDied","Data":"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"}
Nov 02 22:10:33.963869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.963793    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s" event={"ID":"3d9959ae-f7d0-48c3-8370-3647e4ca50e1","Type":"ContainerDied","Data":"4c217ab78eacd5b63cb0942cf2cd1272e28f20f31df97edeeb38f717d69d1330"}
Nov 02 22:10:33.963869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.963811    5580 scope.go:122] "RemoveContainer" containerID="9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"
Nov 02 22:10:33.964312 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.964292    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:33.965613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.965592    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:10:33.975559 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.975536    5580 scope.go:122] "RemoveContainer" containerID="498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"
Nov 02 22:10:33.989904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.989877    5580 scope.go:122] "RemoveContainer" containerID="9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"
Nov 02 22:10:33.990282 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:33.990256    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7\": container with ID starting with 9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7 not found: ID does not exist" containerID="9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"
Nov 02 22:10:33.990375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.990295    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7"} err="failed to get container status \"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7\": rpc error: code = NotFound desc = could not find container \"9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7\": container with ID starting with 9d9b8bea4600e635ac9ce12812b914fe4236239be81e8c17a14f8f3b34c379b7 not found: ID does not exist"
Nov 02 22:10:33.990375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.990315    5580 scope.go:122] "RemoveContainer" containerID="498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"
Nov 02 22:10:33.990552 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:33.990532    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a\": container with ID starting with 498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a not found: ID does not exist" containerID="498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"
Nov 02 22:10:33.990608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:33.990560    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a"} err="failed to get container status \"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a\": rpc error: code = NotFound desc = could not find container \"498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a\": container with ID starting with 498fe99158a2c03dd57dd0a11e3b8dff825fc984001b353a3e7bb88b9879b20a not found: ID does not exist"
Nov 02 22:10:34.252372 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:34.252283    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:10:34.255647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:34.255614    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-588b5c-openshift-default-748f8d69bc-xmr2s"]
Nov 02 22:10:35.595802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:35.595763    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" path="/var/lib/kubelet/pods/3d9959ae-f7d0-48c3-8370-3647e4ca50e1/volumes"
Nov 02 22:10:37.470718 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.470673    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:10:37.471600 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.471577    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:10:37.471707 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.471602    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:10:37.471823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.471809    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:10:37.476810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.476787    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.532145 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.532114    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:10:37.577054 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.577015    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-bdc145-serving-cert\""
Nov 02 22:10:37.604851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.604811    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2cc\" (UniqueName: \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.605090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.605043    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.605090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.605082    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.605196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.605169    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.706624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.706584    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2cc\" (UniqueName: \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.706791 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.706722    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.706791 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.706754    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.706900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.706805    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.706989 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:37.706967    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-serving-cert: secret "maas-api-e2e-api-a-bdc145-serving-cert" not found
Nov 02 22:10:37.707055 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:37.707044    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls podName:ee30fd55-f019-40aa-8326-c2aff76f2951 nodeName:}" failed. No retries permitted until 2026-11-02 22:10:38.207019458 +0000 UTC m=+2136.723515236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls") pod "maas-api-e2e-api-a-bdc145-778d865748-cxhvt" (UID: "ee30fd55-f019-40aa-8326-c2aff76f2951") : secret "maas-api-e2e-api-a-bdc145-serving-cert" not found
Nov 02 22:10:37.707451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.707431    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.709358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.709335    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:37.764710 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:37.764618    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2cc\" (UniqueName: \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:38.224013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:38.219114    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:38.284392 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:38.282071    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") pod \"maas-api-e2e-api-a-bdc145-778d865748-cxhvt\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:38.391601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:38.391555    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:39.053925 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:10:39.053885    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee30fd55_f019_40aa_8326_c2aff76f2951.slice/crio-9be5da8706149e3bf294bfcd509ebb1b7c70a8c93abfc3243ac92db8a78252a6 WatchSource:0}: Error finding container 9be5da8706149e3bf294bfcd509ebb1b7c70a8c93abfc3243ac92db8a78252a6: Status 404 returned error can't find the container with id 9be5da8706149e3bf294bfcd509ebb1b7c70a8c93abfc3243ac92db8a78252a6
Nov 02 22:10:39.095148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:39.095109    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:10:40.036585 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:40.036553    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" event={"ID":"ee30fd55-f019-40aa-8326-c2aff76f2951","Type":"ContainerStarted","Data":"fd23a57cd7baf2b379513149eddf874d15c0b140ae6cd615066ae100efdc2d98"}
Nov 02 22:10:40.036774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:40.036604    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" event={"ID":"ee30fd55-f019-40aa-8326-c2aff76f2951","Type":"ContainerStarted","Data":"9be5da8706149e3bf294bfcd509ebb1b7c70a8c93abfc3243ac92db8a78252a6"}
Nov 02 22:10:40.036774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:40.036638    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:40.184739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:40.184679    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" podStartSLOduration=2.347914097 podStartE2EDuration="3.184660439s" totalImagesPullingTime="836.746342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:40.182393395 +0000 UTC m=+2138.698889171" watchObservedRunningTime="2026-11-02 22:10:40.184660439 +0000 UTC m=+2138.701156277"
Nov 02 22:10:46.050093 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:46.050056    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:10:56.250730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.250695    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:10:56.251307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.250910    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" podUID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" containerName="maas-api" containerID="cri-o://74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446" gracePeriod=30
Nov 02 22:10:56.327206 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:56.327157    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:10:56.573141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.573113    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:10:56.686276 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.686235    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\") pod \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") "
Nov 02 22:10:56.686546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.686528    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\" (UniqueName: \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\") pod \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") "
Nov 02 22:10:56.686775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.686740    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca" pod "ce2b6302-47b8-4974-8b87-71ef778ad3f0" (UID: "ce2b6302-47b8-4974-8b87-71ef778ad3f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:10:56.686850 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.686755    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") pod \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") "
Nov 02 22:10:56.686911 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.686897    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\") pod \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\" (UID: \"ce2b6302-47b8-4974-8b87-71ef778ad3f0\") "
Nov 02 22:10:56.687889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.687860    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2b6302-47b8-4974-8b87-71ef778ad3f0-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:56.693909 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.693875    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls" pod "ce2b6302-47b8-4974-8b87-71ef778ad3f0" (UID: "ce2b6302-47b8-4974-8b87-71ef778ad3f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:56.694198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.694166    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs" pod "ce2b6302-47b8-4974-8b87-71ef778ad3f0" (UID: "ce2b6302-47b8-4974-8b87-71ef778ad3f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:10:56.694520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.694494    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx" pod "ce2b6302-47b8-4974-8b87-71ef778ad3f0" (UID: "ce2b6302-47b8-4974-8b87-71ef778ad3f0"). InnerVolumeSpecName "kube-api-access-cqggx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:10:56.789841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.789743    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqggx\" (UniqueName: \"kubernetes.io/projected/ce2b6302-47b8-4974-8b87-71ef778ad3f0-kube-api-access-cqggx\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:56.789841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.789784    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:56.789841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:56.789796    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2b6302-47b8-4974-8b87-71ef778ad3f0-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:10:57.277478 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.277438    5580 generic.go:356] "Generic (PLEG): container finished" podID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" containerID="74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446" exitCode=0
Nov 02 22:10:57.278279 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.278250    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" event={"ID":"ce2b6302-47b8-4974-8b87-71ef778ad3f0","Type":"ContainerDied","Data":"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"}
Nov 02 22:10:57.278439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.278424    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr" event={"ID":"ce2b6302-47b8-4974-8b87-71ef778ad3f0","Type":"ContainerDied","Data":"1ab473876f52c0759a977f1899aff844eed3c4d18f718872bbb1ca9f1af7dc83"}
Nov 02 22:10:57.278536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.278525    5580 scope.go:122] "RemoveContainer" containerID="74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"
Nov 02 22:10:57.278823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.278799    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"
Nov 02 22:10:57.322574 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.322544    5580 scope.go:122] "RemoveContainer" containerID="74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"
Nov 02 22:10:57.323057 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:10:57.323035    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446\": container with ID starting with 74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446 not found: ID does not exist" containerID="74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"
Nov 02 22:10:57.323171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.323071    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446"} err="failed to get container status \"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446\": rpc error: code = NotFound desc = could not find container \"74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446\": container with ID starting with 74025ed242edfc9848852e5d066472da3f130ae21b69216473c294e1d495d446 not found: ID does not exist"
Nov 02 22:10:57.468143 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.468097    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:10:57.471391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.471369    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fc235-dbddf66b6-mvbpr"]
Nov 02 22:10:57.596103 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:10:57.596017    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" path="/var/lib/kubelet/pods/ce2b6302-47b8-4974-8b87-71ef778ad3f0/volumes"
Nov 02 22:11:07.682628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:07.682589    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:11:16.189731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:16.189632    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:11:21.190461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.190409    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" podUID="b0732474-c0ee-41c3-9e31-a79383d1d776" containerName="payload-pre-processing" containerID="cri-o://120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947" gracePeriod=25
Nov 02 22:11:21.486289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.486262    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:11:21.597195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.597160    5580 generic.go:356] "Generic (PLEG): container finished" podID="b0732474-c0ee-41c3-9e31-a79383d1d776" containerID="120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947" exitCode=0
Nov 02 22:11:21.597423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.597272    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"
Nov 02 22:11:21.598684 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.598651    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" event={"ID":"b0732474-c0ee-41c3-9e31-a79383d1d776","Type":"ContainerDied","Data":"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"}
Nov 02 22:11:21.598843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.598724    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx" event={"ID":"b0732474-c0ee-41c3-9e31-a79383d1d776","Type":"ContainerDied","Data":"cc7ce56c9fd3fe7ac807996173fc4dd6912623ba5616d99705eb29a2ce5c7888"}
Nov 02 22:11:21.598843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.598759    5580 scope.go:122] "RemoveContainer" containerID="120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"
Nov 02 22:11:21.607566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.607535    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\") pod \"b0732474-c0ee-41c3-9e31-a79383d1d776\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") "
Nov 02 22:11:21.607823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.607803    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\" (UniqueName: \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\") pod \"b0732474-c0ee-41c3-9e31-a79383d1d776\" (UID: \"b0732474-c0ee-41c3-9e31-a79383d1d776\") "
Nov 02 22:11:21.607928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.607901    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume" pod "b0732474-c0ee-41c3-9e31-a79383d1d776" (UID: "b0732474-c0ee-41c3-9e31-a79383d1d776"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:11:21.608546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.608517    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0732474-c0ee-41c3-9e31-a79383d1d776-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:11:21.610394 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.610360    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8" pod "b0732474-c0ee-41c3-9e31-a79383d1d776" (UID: "b0732474-c0ee-41c3-9e31-a79383d1d776"). InnerVolumeSpecName "kube-api-access-9d6z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:11:21.612929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.612804    5580 scope.go:122] "RemoveContainer" containerID="120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"
Nov 02 22:11:21.613267 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:11:21.613242    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947\": container with ID starting with 120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947 not found: ID does not exist" containerID="120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"
Nov 02 22:11:21.613368 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.613275    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947"} err="failed to get container status \"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947\": rpc error: code = NotFound desc = could not find container \"120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947\": container with ID starting with 120cd1ce3f74222523cb173b9bd124857c0ef3c63c2380140928a5ad5eb06947 not found: ID does not exist"
Nov 02 22:11:21.711000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:21.710893    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d6z8\" (UniqueName: \"kubernetes.io/projected/b0732474-c0ee-41c3-9e31-a79383d1d776-kube-api-access-9d6z8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:11:22.019297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:22.019167    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:11:22.068402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:22.068351    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-587629-56f7967dd7-5qmfx"]
Nov 02 22:11:23.595894 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:23.595859    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0732474-c0ee-41c3-9e31-a79383d1d776" path="/var/lib/kubelet/pods/b0732474-c0ee-41c3-9e31-a79383d1d776/volumes"
Nov 02 22:11:26.314057 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:11:26.314008    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:11:39.920347 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:39.920302    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:11:44.921055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:44.921006    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" podUID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" containerName="payload-pre-processing" containerID="cri-o://903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c" gracePeriod=25
Nov 02 22:11:45.230298 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.230259    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:11:45.358686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.358653    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\") pod \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") "
Nov 02 22:11:45.358885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.358861    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\" (UniqueName: \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\") pod \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\" (UID: \"72b9229a-f32d-4b46-9247-4f9d32a6bea6\") "
Nov 02 22:11:45.359164 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.359132    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume" pod "72b9229a-f32d-4b46-9247-4f9d32a6bea6" (UID: "72b9229a-f32d-4b46-9247-4f9d32a6bea6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:11:45.360032 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.360002    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72b9229a-f32d-4b46-9247-4f9d32a6bea6-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:11:45.361460 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.361427    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b" pod "72b9229a-f32d-4b46-9247-4f9d32a6bea6" (UID: "72b9229a-f32d-4b46-9247-4f9d32a6bea6"). InnerVolumeSpecName "kube-api-access-c886b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:11:45.461503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.461394    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c886b\" (UniqueName: \"kubernetes.io/projected/72b9229a-f32d-4b46-9247-4f9d32a6bea6-kube-api-access-c886b\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:11:45.833731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.833638    5580 generic.go:356] "Generic (PLEG): container finished" podID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" containerID="903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c" exitCode=0
Nov 02 22:11:45.833731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.833698    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"
Nov 02 22:11:45.833942 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.833698    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" event={"ID":"72b9229a-f32d-4b46-9247-4f9d32a6bea6","Type":"ContainerDied","Data":"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"}
Nov 02 22:11:45.833942 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.833809    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7" event={"ID":"72b9229a-f32d-4b46-9247-4f9d32a6bea6","Type":"ContainerDied","Data":"e5d9ae6328f3e733af8c1c414a4f0308f0540a1f39629072da0513d138f9e6dd"}
Nov 02 22:11:45.833942 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.833837    5580 scope.go:122] "RemoveContainer" containerID="903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"
Nov 02 22:11:45.844857 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.844836    5580 scope.go:122] "RemoveContainer" containerID="903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"
Nov 02 22:11:45.845272 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:11:45.845248    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c\": container with ID starting with 903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c not found: ID does not exist" containerID="903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"
Nov 02 22:11:45.845372 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.845282    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c"} err="failed to get container status \"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c\": rpc error: code = NotFound desc = could not find container \"903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c\": container with ID starting with 903c45f007aec988bf450e391dda2a35c4b9a9f09f437b55fbf6c313b2fb232c not found: ID does not exist"
Nov 02 22:11:45.997227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:45.997183    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:11:46.047944 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:46.047902    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-17435d-558569f69f-kzhj7"]
Nov 02 22:11:47.596111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:47.596069    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" path="/var/lib/kubelet/pods/72b9229a-f32d-4b46-9247-4f9d32a6bea6/volumes"
Nov 02 22:11:48.864762 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.864724    5580 generic.go:356] "Generic (PLEG): container finished" podID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerID="7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f" exitCode=0
Nov 02 22:11:48.865327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.864795    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerDied","Data":"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"}
Nov 02 22:11:48.865327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.865306    5580 scope.go:122] "RemoveContainer" containerID="7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"
Nov 02 22:11:48.867320 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.867286    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:11:48.867411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.867367    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:11:48.867411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:48.867406    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:11:49.358402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:49.358363    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:11:49.874639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:49.874599    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerStarted","Data":"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"}
Nov 02 22:11:50.174327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:50.174291    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:11:50.175720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:50.175684    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:50.175885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:50.175755    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:51.175183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:51.175143    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:51.175604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:51.175206    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:52.175449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:52.175410    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:52.175863 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:52.175498    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:53.174895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:53.174855    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:53.175122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:53.174916    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:54.174574 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:54.174541    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:54.175061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:54.174590    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:55.174925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:55.174881    5580 patch_prober.go:29] interesting pod/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 02 22:11:55.175391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:55.174945    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 02 22:11:56.202912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:56.201087    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:11:56.202912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:56.202740    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:11:56.222041 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:11:56.220269    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:11:56.349351 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:11:56.349306    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:12:02.146518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.146480    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:12:02.147004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.146752    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" podUID="c6bf5b71-fcec-406b-be53-9edacc5fad53" containerName="maas-api" containerID="cri-o://93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797" gracePeriod=30
Nov 02 22:12:02.482635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.482604    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:12:02.626541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.626506    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") pod \"c6bf5b71-fcec-406b-be53-9edacc5fad53\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") "
Nov 02 22:12:02.626761 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.626602    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\") pod \"c6bf5b71-fcec-406b-be53-9edacc5fad53\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") "
Nov 02 22:12:02.626831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.626792    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\" (UniqueName: \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\") pod \"c6bf5b71-fcec-406b-be53-9edacc5fad53\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") "
Nov 02 22:12:02.626887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.626875    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\") pod \"c6bf5b71-fcec-406b-be53-9edacc5fad53\" (UID: \"c6bf5b71-fcec-406b-be53-9edacc5fad53\") "
Nov 02 22:12:02.628101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.628050    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca" pod "c6bf5b71-fcec-406b-be53-9edacc5fad53" (UID: "c6bf5b71-fcec-406b-be53-9edacc5fad53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:02.633458 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.633411    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs" pod "c6bf5b71-fcec-406b-be53-9edacc5fad53" (UID: "c6bf5b71-fcec-406b-be53-9edacc5fad53"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:12:02.633670 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.633470    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls" pod "c6bf5b71-fcec-406b-be53-9edacc5fad53" (UID: "c6bf5b71-fcec-406b-be53-9edacc5fad53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:12:02.634543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.634165    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht" pod "c6bf5b71-fcec-406b-be53-9edacc5fad53" (UID: "c6bf5b71-fcec-406b-be53-9edacc5fad53"). InnerVolumeSpecName "kube-api-access-9j9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:02.730322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.730222    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j9ht\" (UniqueName: \"kubernetes.io/projected/c6bf5b71-fcec-406b-be53-9edacc5fad53-kube-api-access-9j9ht\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:02.730555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.730536    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf5b71-fcec-406b-be53-9edacc5fad53-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:02.730668 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.730655    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:02.730768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:02.730756    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6bf5b71-fcec-406b-be53-9edacc5fad53-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:03.005660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.005564    5580 generic.go:356] "Generic (PLEG): container finished" podID="c6bf5b71-fcec-406b-be53-9edacc5fad53" containerID="93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797" exitCode=0
Nov 02 22:12:03.005660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.005617    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"
Nov 02 22:12:03.005660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.005645    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" event={"ID":"c6bf5b71-fcec-406b-be53-9edacc5fad53","Type":"ContainerDied","Data":"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"}
Nov 02 22:12:03.005939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.005709    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx" event={"ID":"c6bf5b71-fcec-406b-be53-9edacc5fad53","Type":"ContainerDied","Data":"26d954defda8e9fc5ddf932337327eef33fa7ec1a724592e01898b00059cca07"}
Nov 02 22:12:03.005939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.005738    5580 scope.go:122] "RemoveContainer" containerID="93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"
Nov 02 22:12:03.017518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.017495    5580 scope.go:122] "RemoveContainer" containerID="93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"
Nov 02 22:12:03.017879 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:03.017855    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797\": container with ID starting with 93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797 not found: ID does not exist" containerID="93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"
Nov 02 22:12:03.017951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.017894    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797"} err="failed to get container status \"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797\": rpc error: code = NotFound desc = could not find container \"93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797\": container with ID starting with 93b5bb7bbadd994e89ebcbf87aa73949ca6b9378340ab6648fedd4926630e797 not found: ID does not exist"
Nov 02 22:12:03.212986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.212621    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:12:03.230174 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.228902    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2050f3-6d5c95598d-wngmx"]
Nov 02 22:12:03.596603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:03.596569    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6bf5b71-fcec-406b-be53-9edacc5fad53" path="/var/lib/kubelet/pods/c6bf5b71-fcec-406b-be53-9edacc5fad53/volumes"
Nov 02 22:12:08.581362 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.581322    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:12:08.582272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582246    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b0732474-c0ee-41c3-9e31-a79383d1d776" containerName="payload-pre-processing"
Nov 02 22:12:08.582272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582271    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0732474-c0ee-41c3-9e31-a79383d1d776" containerName="payload-pre-processing"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582312    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582320    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582337    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c6bf5b71-fcec-406b-be53-9edacc5fad53" containerName="maas-api"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582345    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6bf5b71-fcec-406b-be53-9edacc5fad53" containerName="maas-api"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582364    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" containerName="maas-api"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582371    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" containerName="maas-api"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582402    5580 cpu_manager.go:414] "containerMap: removing container" podUID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" containerName="payload-pre-processing"
Nov 02 22:12:08.582453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582410    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" containerName="payload-pre-processing"
Nov 02 22:12:08.582829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582581    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2b6302-47b8-4974-8b87-71ef778ad3f0" containerName="maas-api"
Nov 02 22:12:08.582829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582603    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="72b9229a-f32d-4b46-9247-4f9d32a6bea6" containerName="payload-pre-processing"
Nov 02 22:12:08.582829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582622    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0732474-c0ee-41c3-9e31-a79383d1d776" containerName="payload-pre-processing"
Nov 02 22:12:08.582829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582635    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d9959ae-f7d0-48c3-8370-3647e4ca50e1" containerName="istio-proxy"
Nov 02 22:12:08.582829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.582645    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6bf5b71-fcec-406b-be53-9edacc5fad53" containerName="maas-api"
Nov 02 22:12:08.586136 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.586105    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.638209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.638176    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1789c6-serving-cert\""
Nov 02 22:12:08.692750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.692707    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:12:08.725476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.725430    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qskx7\" (UniqueName: \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.725698 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.725623    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.725780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.725725    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.725990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.725946    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.827661 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.827611    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qskx7\" (UniqueName: \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.827858 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.827730    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.827858 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.827807    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.828011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.827988    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.828266 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:08.828243    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-serving-cert: secret "maas-api-e2e-inf-a-1789c6-serving-cert" not found
Nov 02 22:12:08.828464 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:08.828331    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls podName:705e02c7-d567-4802-ae53-a1a83f630d9e nodeName:}" failed. No retries permitted until 2026-11-02 22:12:09.32831058 +0000 UTC m=+2227.844806348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls") pod "maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" (UID: "705e02c7-d567-4802-ae53-a1a83f630d9e") : secret "maas-api-e2e-inf-a-1789c6-serving-cert" not found
Nov 02 22:12:08.828834 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.828808    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.830748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.830719    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:08.885833 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:08.885746    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qskx7\" (UniqueName: \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:09.337351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:09.337299    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:09.340032 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:09.340001    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:09.500083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:09.500023    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:09.889314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:09.889278    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:12:09.890655 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:12:09.890626    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705e02c7_d567_4802_ae53_a1a83f630d9e.slice/crio-a2fb4a6a376514957f5d5c67affa724d6e9fdcf86806f4b46f8423c05c3991fb WatchSource:0}: Error finding container a2fb4a6a376514957f5d5c67affa724d6e9fdcf86806f4b46f8423c05c3991fb: Status 404 returned error can't find the container with id a2fb4a6a376514957f5d5c67affa724d6e9fdcf86806f4b46f8423c05c3991fb
Nov 02 22:12:10.059266 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:10.059223    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" event={"ID":"705e02c7-d567-4802-ae53-a1a83f630d9e","Type":"ContainerStarted","Data":"a2fb4a6a376514957f5d5c67affa724d6e9fdcf86806f4b46f8423c05c3991fb"}
Nov 02 22:12:11.070755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:11.070661    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" event={"ID":"705e02c7-d567-4802-ae53-a1a83f630d9e","Type":"ContainerStarted","Data":"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"}
Nov 02 22:12:11.071233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:11.070865    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:11.185384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:11.185328    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" podStartSLOduration=2.350358855 podStartE2EDuration="3.185313907s" totalImagesPullingTime="834.955052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:11.184214071 +0000 UTC m=+2229.700709845" watchObservedRunningTime="2026-11-02 22:12:11.185313907 +0000 UTC m=+2229.701809681"
Nov 02 22:12:17.083629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:17.083599    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:12:18.410539 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:18.410489    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:12:18.530634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:18.530598    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:12:21.261803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:21.261766    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:12:23.424726 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.424682    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" podUID="92249a10-e668-4fcc-9bbf-8770552b2563" containerName="payload-processing" containerID="cri-o://f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5" gracePeriod=25
Nov 02 22:12:23.531922 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.531807    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" podUID="1f474e96-2212-474d-b7e4-b8813cdc398e" containerName="payload-pre-processing" containerID="cri-o://58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527" gracePeriod=25
Nov 02 22:12:23.729658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.729631    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:12:23.835224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.835187    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\" (UniqueName: \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\") pod \"92249a10-e668-4fcc-9bbf-8770552b2563\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") "
Nov 02 22:12:23.835477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.835454    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\") pod \"92249a10-e668-4fcc-9bbf-8770552b2563\" (UID: \"92249a10-e668-4fcc-9bbf-8770552b2563\") "
Nov 02 22:12:23.836024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.835926    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume" pod "92249a10-e668-4fcc-9bbf-8770552b2563" (UID: "92249a10-e668-4fcc-9bbf-8770552b2563"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:23.836159 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.836138    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92249a10-e668-4fcc-9bbf-8770552b2563-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:23.838606 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.838580    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k" pod "92249a10-e668-4fcc-9bbf-8770552b2563" (UID: "92249a10-e668-4fcc-9bbf-8770552b2563"). InnerVolumeSpecName "kube-api-access-c578k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:23.846625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.843494    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:12:23.936813 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.936773    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\") pod \"1f474e96-2212-474d-b7e4-b8813cdc398e\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") "
Nov 02 22:12:23.937072 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.936932    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\" (UniqueName: \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\") pod \"1f474e96-2212-474d-b7e4-b8813cdc398e\" (UID: \"1f474e96-2212-474d-b7e4-b8813cdc398e\") "
Nov 02 22:12:23.937276 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.937250    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume" pod "1f474e96-2212-474d-b7e4-b8813cdc398e" (UID: "1f474e96-2212-474d-b7e4-b8813cdc398e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:23.937604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.937584    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c578k\" (UniqueName: \"kubernetes.io/projected/92249a10-e668-4fcc-9bbf-8770552b2563-kube-api-access-c578k\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:23.937671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.937609    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f474e96-2212-474d-b7e4-b8813cdc398e-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:23.939146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:23.939110    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh" pod "1f474e96-2212-474d-b7e4-b8813cdc398e" (UID: "1f474e96-2212-474d-b7e4-b8813cdc398e"). InnerVolumeSpecName "kube-api-access-tmhbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:24.039363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.039255    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmhbh\" (UniqueName: \"kubernetes.io/projected/1f474e96-2212-474d-b7e4-b8813cdc398e-kube-api-access-tmhbh\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:24.169494 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.169448    5580 generic.go:356] "Generic (PLEG): container finished" podID="92249a10-e668-4fcc-9bbf-8770552b2563" containerID="f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5" exitCode=0
Nov 02 22:12:24.169494 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.169487    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" event={"ID":"92249a10-e668-4fcc-9bbf-8770552b2563","Type":"ContainerDied","Data":"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"}
Nov 02 22:12:24.169760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.169529    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"
Nov 02 22:12:24.169760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.169574    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l" event={"ID":"92249a10-e668-4fcc-9bbf-8770552b2563","Type":"ContainerDied","Data":"845509d5a458d9803a520c165137736481e2bcf8c8d9a2b803aa13c15068c876"}
Nov 02 22:12:24.169760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.169605    5580 scope.go:122] "RemoveContainer" containerID="f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"
Nov 02 22:12:24.171274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.171248    5580 generic.go:356] "Generic (PLEG): container finished" podID="1f474e96-2212-474d-b7e4-b8813cdc398e" containerID="58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527" exitCode=0
Nov 02 22:12:24.171385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.171371    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"
Nov 02 22:12:24.171530 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.171508    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" event={"ID":"1f474e96-2212-474d-b7e4-b8813cdc398e","Type":"ContainerDied","Data":"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"}
Nov 02 22:12:24.171589 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.171559    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq" event={"ID":"1f474e96-2212-474d-b7e4-b8813cdc398e","Type":"ContainerDied","Data":"aae90871460f3e25eed2c22140b1036066917c354387c3925b6be8f761395dd2"}
Nov 02 22:12:24.183507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.183481    5580 scope.go:122] "RemoveContainer" containerID="f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"
Nov 02 22:12:24.183865 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:24.183836    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5\": container with ID starting with f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5 not found: ID does not exist" containerID="f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"
Nov 02 22:12:24.183944 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.183882    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5"} err="failed to get container status \"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5\": rpc error: code = NotFound desc = could not find container \"f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5\": container with ID starting with f9454c72e735710e8ef2cd5d40e754735b0fe5c658f74ff999aa18e4859a77d5 not found: ID does not exist"
Nov 02 22:12:24.183944 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.183911    5580 scope.go:122] "RemoveContainer" containerID="58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"
Nov 02 22:12:24.199713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.199691    5580 scope.go:122] "RemoveContainer" containerID="58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"
Nov 02 22:12:24.199990 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:24.199929    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527\": container with ID starting with 58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527 not found: ID does not exist" containerID="58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"
Nov 02 22:12:24.200078 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.199982    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527"} err="failed to get container status \"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527\": rpc error: code = NotFound desc = could not find container \"58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527\": container with ID starting with 58c0b007a86d88c2720f7d5e25c1c4bc50bf7785f03a412165793350d6f8d527 not found: ID does not exist"
Nov 02 22:12:24.296283 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.296191    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:12:24.332673 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.332633    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:12:24.338085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.338051    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bdc145-5594f9f759-pg8fq"]
Nov 02 22:12:24.448753 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.448726    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:12:24.452205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:24.452175    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bdc145-6dc94c4b85-t5x9l"]
Nov 02 22:12:25.625025 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:25.624939    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f474e96-2212-474d-b7e4-b8813cdc398e" path="/var/lib/kubelet/pods/1f474e96-2212-474d-b7e4-b8813cdc398e/volumes"
Nov 02 22:12:25.625633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:25.625611    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92249a10-e668-4fcc-9bbf-8770552b2563" path="/var/lib/kubelet/pods/92249a10-e668-4fcc-9bbf-8770552b2563/volumes"
Nov 02 22:12:26.313249 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:26.313198    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:12:29.297256 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.297176    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" podUID="d60eb4a7-af94-4e56-89d7-bf3202287774" containerName="payload-processing" containerID="cri-o://01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049" gracePeriod=25
Nov 02 22:12:29.599621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.599589    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:12:29.658782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.658744    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\" (UniqueName: \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\") pod \"d60eb4a7-af94-4e56-89d7-bf3202287774\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") "
Nov 02 22:12:29.659053 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.658832    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\") pod \"d60eb4a7-af94-4e56-89d7-bf3202287774\" (UID: \"d60eb4a7-af94-4e56-89d7-bf3202287774\") "
Nov 02 22:12:29.659271 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.659242    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume" pod "d60eb4a7-af94-4e56-89d7-bf3202287774" (UID: "d60eb4a7-af94-4e56-89d7-bf3202287774"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:29.659869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.659846    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d60eb4a7-af94-4e56-89d7-bf3202287774-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:29.661414 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.661385    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8" pod "d60eb4a7-af94-4e56-89d7-bf3202287774" (UID: "d60eb4a7-af94-4e56-89d7-bf3202287774"). InnerVolumeSpecName "kube-api-access-xtpn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:29.762577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:29.762537    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtpn8\" (UniqueName: \"kubernetes.io/projected/d60eb4a7-af94-4e56-89d7-bf3202287774-kube-api-access-xtpn8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:30.219421 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.219385    5580 generic.go:356] "Generic (PLEG): container finished" podID="d60eb4a7-af94-4e56-89d7-bf3202287774" containerID="01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049" exitCode=0
Nov 02 22:12:30.219621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.219431    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"
Nov 02 22:12:30.219621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.219470    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" event={"ID":"d60eb4a7-af94-4e56-89d7-bf3202287774","Type":"ContainerDied","Data":"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"}
Nov 02 22:12:30.219621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.219527    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t" event={"ID":"d60eb4a7-af94-4e56-89d7-bf3202287774","Type":"ContainerDied","Data":"e7d273b38d784a8d0612d360a7491a7aecfcebdd80da98b191b37c0006c6566e"}
Nov 02 22:12:30.219621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.219552    5580 scope.go:122] "RemoveContainer" containerID="01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"
Nov 02 22:12:30.230996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.230935    5580 scope.go:122] "RemoveContainer" containerID="01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"
Nov 02 22:12:30.231292 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:30.231273    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049\": container with ID starting with 01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049 not found: ID does not exist" containerID="01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"
Nov 02 22:12:30.231399 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.231303    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049"} err="failed to get container status \"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049\": rpc error: code = NotFound desc = could not find container \"01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049\": container with ID starting with 01d2f8e8459678db5098997e99abc9c6d4ef0c8967b2ec04174fc25967d45049 not found: ID does not exist"
Nov 02 22:12:30.384925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.384886    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:12:30.432842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:30.432794    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d1c62e-6c4755dc85-mww7t"]
Nov 02 22:12:31.597058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.597023    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d60eb4a7-af94-4e56-89d7-bf3202287774" path="/var/lib/kubelet/pods/d60eb4a7-af94-4e56-89d7-bf3202287774/volumes"
Nov 02 22:12:31.597536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.597356    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 22:12:31.597590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.597564    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy" containerID="cri-o://d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066" gracePeriod=30
Nov 02 22:12:31.963122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.963083    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:12:31.963329 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.963292    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" containerID="cri-o://056012b158459e17a380107beed2497fdaeccb94980825ad067ef2fe2d63dd78" gracePeriod=30
Nov 02 22:12:31.976411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.976376    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:12:31.976636 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.976597    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" containerID="cri-o://00c13ed93046e9de34e53db6349345543e54778fea7bb6e8501b1e10723a5ca3" gracePeriod=30
Nov 02 22:12:31.981364 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.981335    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:12:31.981534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:31.981514    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" containerID="cri-o://b24a576683c215d918e952d23aa3404aa7b91ca92d2840a915b962303af2b96a" gracePeriod=30
Nov 02 22:12:34.140087 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:34.140040    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:12:34.140502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:34.140129    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:12:36.193486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:36.193449    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:12:36.193887 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:36.193522    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:12:36.241227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:36.241188    5580 patch_prober.go:29] interesting pod/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:12:36.241389 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:36.241271    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:12:36.914799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:36.914766    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.099930    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100054    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100148    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100196    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100245    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100363    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100474    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100511    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.100586    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\") pod \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\" (UID: \"7e14dba8-4d31-40b5-b273-67ca8d96ddb9\") "
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.102358    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.102777    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.103254    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.103321    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.104918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.103557    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:37.109739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.109712    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "kube-api-access-zhfr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.109882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.109797    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.118581 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.118532    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.128039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.127798    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo" pod "7e14dba8-4d31-40b5-b273-67ca8d96ddb9" (UID: "7e14dba8-4d31-40b5-b273-67ca8d96ddb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:37.202484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.202449    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhfr6\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-kube-api-access-zhfr6\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.202927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.202911    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203033    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203058    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203072    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203087    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203101    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203115    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.203212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.203127    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e14dba8-4d31-40b5-b273-67ca8d96ddb9-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.389031 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.388897    5580 generic.go:356] "Generic (PLEG): container finished" podID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerID="d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066" exitCode=0
Nov 02 22:12:37.389230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.389215    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"
Nov 02 22:12:37.390503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.390226    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" event={"ID":"7e14dba8-4d31-40b5-b273-67ca8d96ddb9","Type":"ContainerDied","Data":"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"}
Nov 02 22:12:37.390503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.390293    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88" event={"ID":"7e14dba8-4d31-40b5-b273-67ca8d96ddb9","Type":"ContainerDied","Data":"c6e6acfa35ca84fd6b4ffedfb2cf23de594b83c46a62a2bcbb68fc17732842cd"}
Nov 02 22:12:37.390503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.390326    5580 scope.go:122] "RemoveContainer" containerID="d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"
Nov 02 22:12:37.393144 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.392471    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:12:37.394416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.394378    5580 generic.go:356] "Generic (PLEG): container finished" podID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerID="b24a576683c215d918e952d23aa3404aa7b91ca92d2840a915b962303af2b96a" exitCode=0
Nov 02 22:12:37.394531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.394485    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" event={"ID":"04d6874f-bb2c-4142-9793-481d0b108b6b","Type":"ContainerDied","Data":"b24a576683c215d918e952d23aa3404aa7b91ca92d2840a915b962303af2b96a"}
Nov 02 22:12:37.396449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.396425    5580 generic.go:356] "Generic (PLEG): container finished" podID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerID="056012b158459e17a380107beed2497fdaeccb94980825ad067ef2fe2d63dd78" exitCode=0
Nov 02 22:12:37.396558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.396500    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm" event={"ID":"b9a3988e-fac7-4132-93b5-6f97f7ac7c32","Type":"ContainerDied","Data":"056012b158459e17a380107beed2497fdaeccb94980825ad067ef2fe2d63dd78"}
Nov 02 22:12:37.396628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.396570    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"
Nov 02 22:12:37.398690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.398549    5580 generic.go:356] "Generic (PLEG): container finished" podID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerID="00c13ed93046e9de34e53db6349345543e54778fea7bb6e8501b1e10723a5ca3" exitCode=0
Nov 02 22:12:37.398690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.398628    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" event={"ID":"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e","Type":"ContainerDied","Data":"00c13ed93046e9de34e53db6349345543e54778fea7bb6e8501b1e10723a5ca3"}
Nov 02 22:12:37.413083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.413062    5580 scope.go:122] "RemoveContainer" containerID="d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"
Nov 02 22:12:37.417242 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:37.417212    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066\": container with ID starting with d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066 not found: ID does not exist" containerID="d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"
Nov 02 22:12:37.417443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.417406    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066"} err="failed to get container status \"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066\": rpc error: code = NotFound desc = could not find container \"d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066\": container with ID starting with d47ff3e109921bd042f138fb36269f313de76645e827d3833a8ee20ee5606066 not found: ID does not exist"
Nov 02 22:12:37.417443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.417444    5580 scope.go:122] "RemoveContainer" containerID="056012b158459e17a380107beed2497fdaeccb94980825ad067ef2fe2d63dd78"
Nov 02 22:12:37.432074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.432037    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:12:37.481554 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.481526    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:12:37.546115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546078    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.546299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546125    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.546299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546146    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.546299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546211    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546284    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.546502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546370    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.546502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546403    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546422    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.546502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546444    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.546502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546488    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546521    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546549    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546586    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546616    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546649    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546672    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.546787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546734    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546795    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546797    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546828    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546859    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546864    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546886    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.546974    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547014    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547047    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547084    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547113    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547150    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547201    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547205    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\") pod \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\" (UID: \"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e\") "
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547253    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\") pod \"04d6874f-bb2c-4142-9793-481d0b108b6b\" (UID: \"04d6874f-bb2c-4142-9793-481d0b108b6b\") "
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547322    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\") pod \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\" (UID: \"b9a3988e-fac7-4132-93b5-6f97f7ac7c32\") "
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547464    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547477    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547509    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547678    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547697    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547907    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.547950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.547914    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.548570 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.548150    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.549218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.549166    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:37.549739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.549715    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.549885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.549870    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.549988    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550086    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550200    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550304    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550396    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550475    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550555    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d6874f-bb2c-4142-9793-481d0b108b6b-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550655 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550642    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550746    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550845    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.550976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550947    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.551088 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.551066    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.551088 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.551083    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.551088 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.550906    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.552548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.551800    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.552548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.552106    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.552548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.552305    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.552548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.552512    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "kube-api-access-m2z5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.552810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.552758    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "kube-api-access-q49t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.552912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.552884    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:37.553069 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.553045    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:37.553357 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.553325    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:37.554402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.554366    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token" pod "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" (UID: "5c8dc48b-3c3a-4def-abf8-1638d8e47a4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.554950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.554925    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo" pod "04d6874f-bb2c-4142-9793-481d0b108b6b" (UID: "04d6874f-bb2c-4142-9793-481d0b108b6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:37.555303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.555283    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8" pod "b9a3988e-fac7-4132-93b5-6f97f7ac7c32" (UID: "b9a3988e-fac7-4132-93b5-6f97f7ac7c32"). InnerVolumeSpecName "kube-api-access-mrnp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:37.653851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653758    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.653851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653808    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.653851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653830    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.653851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653845    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653865    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrnp8\" (UniqueName: \"kubernetes.io/projected/b9a3988e-fac7-4132-93b5-6f97f7ac7c32-kube-api-access-mrnp8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653890    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653909    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q49t6\" (UniqueName: \"kubernetes.io/projected/04d6874f-bb2c-4142-9793-481d0b108b6b-kube-api-access-q49t6\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653925    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653940    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2z5d\" (UniqueName: \"kubernetes.io/projected/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-kube-api-access-m2z5d\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653973    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.653992    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d6874f-bb2c-4142-9793-481d0b108b6b-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.654195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.654007    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:37.837591 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.837542    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 22:12:37.842120 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:37.842087    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5hf88"]
Nov 02 22:12:38.064701 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.064663    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:12:38.067879 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.067845    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-ndcdm"]
Nov 02 22:12:38.406785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.406672    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm" event={"ID":"04d6874f-bb2c-4142-9793-481d0b108b6b","Type":"ContainerDied","Data":"4bceadb05cb741839cc33e2df22daeb2b954e2e31dcfb57ea89842f3f3ebf20e"}
Nov 02 22:12:38.406785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.406703    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"
Nov 02 22:12:38.406785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.406785    5580 scope.go:122] "RemoveContainer" containerID="b24a576683c215d918e952d23aa3404aa7b91ca92d2840a915b962303af2b96a"
Nov 02 22:12:38.409993 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.409932    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"
Nov 02 22:12:38.410157 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.409931    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj" event={"ID":"5c8dc48b-3c3a-4def-abf8-1638d8e47a4e","Type":"ContainerDied","Data":"608245cfd1230a4c7913b7065fbf2042da18cc6c943bd3b6100afb5e37c51a06"}
Nov 02 22:12:38.418675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.418652    5580 scope.go:122] "RemoveContainer" containerID="00c13ed93046e9de34e53db6349345543e54778fea7bb6e8501b1e10723a5ca3"
Nov 02 22:12:38.571534 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.571504    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:12:38.574642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.574612    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-pjmzj"]
Nov 02 22:12:38.683905 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.683865    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:12:38.687030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:38.686997    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2be7cc3-openshift-default-6469c596f-4kxbm"]
Nov 02 22:12:39.595822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:39.595785    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" path="/var/lib/kubelet/pods/04d6874f-bb2c-4142-9793-481d0b108b6b/volumes"
Nov 02 22:12:39.596682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:39.596601    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" path="/var/lib/kubelet/pods/5c8dc48b-3c3a-4def-abf8-1638d8e47a4e/volumes"
Nov 02 22:12:39.597547 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:39.597518    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" path="/var/lib/kubelet/pods/7e14dba8-4d31-40b5-b273-67ca8d96ddb9/volumes"
Nov 02 22:12:39.598675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:39.598653    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" path="/var/lib/kubelet/pods/b9a3988e-fac7-4132-93b5-6f97f7ac7c32/volumes"
Nov 02 22:12:45.797803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:45.797768    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 22:12:45.798301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:45.798110    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" containerID="cri-o://d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0" gracePeriod=30
Nov 02 22:12:46.681154 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:46.681112    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:12:46.906232 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:46.906194    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:12:48.024685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:48.024648    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:12:48.025325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:48.024906    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" containerID="cri-o://32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a" gracePeriod=30
Nov 02 22:12:49.695259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:49.695218    5580 patch_prober.go:29] interesting pod/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:12:49.695690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:49.695289    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:12:50.034861 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:50.034766    5580 patch_prober.go:29] interesting pod/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:12:50.034861 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:50.034828    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:12:51.076903 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.076834    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 22:12:51.183173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183132    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183214    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183243    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183287    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183409    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183478    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183521    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183548    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183566    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:51.183699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183592    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:51.183699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183624    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\") pod \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\" (UID: \"bfb4a535-9ea2-4ddb-a245-d374744a1c4a\") "
Nov 02 22:12:51.183894 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183733    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:51.184000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.183974    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:51.184055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184018    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:51.184334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184317    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.184470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184335    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.184470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184350    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.184470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184363    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.184470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.184376    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.185928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.185903    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:51.185928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.185912    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:51.186261 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.186244    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:51.186335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.186314    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2" pod "bfb4a535-9ea2-4ddb-a245-d374744a1c4a" (UID: "bfb4a535-9ea2-4ddb-a245-d374744a1c4a"). InnerVolumeSpecName "kube-api-access-78mk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:51.285438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.285391    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.285438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.285428    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.285438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.285438    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.285438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.285447    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78mk2\" (UniqueName: \"kubernetes.io/projected/bfb4a535-9ea2-4ddb-a245-d374744a1c4a-kube-api-access-78mk2\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:51.528496 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.528457    5580 generic.go:356] "Generic (PLEG): container finished" podID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerID="d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0" exitCode=0
Nov 02 22:12:51.528734 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.528510    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"
Nov 02 22:12:51.528734 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.528545    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" event={"ID":"bfb4a535-9ea2-4ddb-a245-d374744a1c4a","Type":"ContainerDied","Data":"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"}
Nov 02 22:12:51.528734 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.528617    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk" event={"ID":"bfb4a535-9ea2-4ddb-a245-d374744a1c4a","Type":"ContainerDied","Data":"c30f77688f6008dc68e16bfc55531b4a4c6b4561373500e9bd9a203034d8b3f1"}
Nov 02 22:12:51.528734 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.528647    5580 scope.go:122] "RemoveContainer" containerID="d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"
Nov 02 22:12:51.540927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.540906    5580 scope.go:122] "RemoveContainer" containerID="d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"
Nov 02 22:12:51.541343 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:51.541321    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0\": container with ID starting with d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0 not found: ID does not exist" containerID="d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"
Nov 02 22:12:51.541398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.541356    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0"} err="failed to get container status \"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0\": rpc error: code = NotFound desc = could not find container \"d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0\": container with ID starting with d3d26087912df2969af692180f91570373db151400bafcba7900ea289d7e39c0 not found: ID does not exist"
Nov 02 22:12:51.681700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.681650    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" podUID="86701424-0da1-4aed-8f2d-cbe63230198e" containerName="payload-processing" containerID="cri-o://408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1" gracePeriod=25
Nov 02 22:12:51.693941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.693906    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 22:12:51.742738 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.742599    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-ntdtk"]
Nov 02 22:12:51.917649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:51.917602    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" podUID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" containerName="payload-pre-processing" containerID="cri-o://a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721" gracePeriod=25
Nov 02 22:12:52.055455 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.055384    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:12:52.174709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.174673    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:12:52.199274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.199238    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\" (UniqueName: \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\") pod \"86701424-0da1-4aed-8f2d-cbe63230198e\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") "
Nov 02 22:12:52.200066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.199545    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\") pod \"86701424-0da1-4aed-8f2d-cbe63230198e\" (UID: \"86701424-0da1-4aed-8f2d-cbe63230198e\") "
Nov 02 22:12:52.200066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.200015    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume" pod "86701424-0da1-4aed-8f2d-cbe63230198e" (UID: "86701424-0da1-4aed-8f2d-cbe63230198e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:52.201042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.200987    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86701424-0da1-4aed-8f2d-cbe63230198e-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:52.201803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.201739    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf" pod "86701424-0da1-4aed-8f2d-cbe63230198e" (UID: "86701424-0da1-4aed-8f2d-cbe63230198e"). InnerVolumeSpecName "kube-api-access-rvzxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:52.301752 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.301719    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\" (UniqueName: \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\") pod \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") "
Nov 02 22:12:52.301917 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.301871    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\") pod \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\" (UID: \"3d213f6c-44ab-454a-adcd-90e7dea9a92e\") "
Nov 02 22:12:52.302287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.302254    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume" pod "3d213f6c-44ab-454a-adcd-90e7dea9a92e" (UID: "3d213f6c-44ab-454a-adcd-90e7dea9a92e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:52.302434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.302380    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvzxf\" (UniqueName: \"kubernetes.io/projected/86701424-0da1-4aed-8f2d-cbe63230198e-kube-api-access-rvzxf\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:52.302434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.302403    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d213f6c-44ab-454a-adcd-90e7dea9a92e-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:52.304030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.304004    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd" pod "3d213f6c-44ab-454a-adcd-90e7dea9a92e" (UID: "3d213f6c-44ab-454a-adcd-90e7dea9a92e"). InnerVolumeSpecName "kube-api-access-xd2pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:52.404022 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.403884    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd2pd\" (UniqueName: \"kubernetes.io/projected/3d213f6c-44ab-454a-adcd-90e7dea9a92e-kube-api-access-xd2pd\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:52.539852 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.539816    5580 generic.go:356] "Generic (PLEG): container finished" podID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" containerID="a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721" exitCode=0
Nov 02 22:12:52.540101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.539865    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"
Nov 02 22:12:52.540101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.539897    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" event={"ID":"3d213f6c-44ab-454a-adcd-90e7dea9a92e","Type":"ContainerDied","Data":"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"}
Nov 02 22:12:52.540101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.539970    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l" event={"ID":"3d213f6c-44ab-454a-adcd-90e7dea9a92e","Type":"ContainerDied","Data":"8f944e421613a9a1504cb26b56423fc7329f390817e65741ae9e0d113f1aa87c"}
Nov 02 22:12:52.540101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.539984    5580 scope.go:122] "RemoveContainer" containerID="a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"
Nov 02 22:12:52.541501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.541481    5580 generic.go:356] "Generic (PLEG): container finished" podID="86701424-0da1-4aed-8f2d-cbe63230198e" containerID="408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1" exitCode=0
Nov 02 22:12:52.541608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.541524    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"
Nov 02 22:12:52.541608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.541556    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" event={"ID":"86701424-0da1-4aed-8f2d-cbe63230198e","Type":"ContainerDied","Data":"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"}
Nov 02 22:12:52.541608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.541603    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm" event={"ID":"86701424-0da1-4aed-8f2d-cbe63230198e","Type":"ContainerDied","Data":"423e6a00c85ded7880743a40b4cddd814ccaec2c20c80b321c1dc791be9c8b67"}
Nov 02 22:12:52.552811 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.552788    5580 scope.go:122] "RemoveContainer" containerID="a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"
Nov 02 22:12:52.553187 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:52.553167    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721\": container with ID starting with a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721 not found: ID does not exist" containerID="a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"
Nov 02 22:12:52.553290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.553198    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721"} err="failed to get container status \"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721\": rpc error: code = NotFound desc = could not find container \"a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721\": container with ID starting with a72f2764007546ca5de92b23381d94d885e72601c285eda0eb0f97571eb85721 not found: ID does not exist"
Nov 02 22:12:52.553290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.553225    5580 scope.go:122] "RemoveContainer" containerID="408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"
Nov 02 22:12:52.564337 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.564316    5580 scope.go:122] "RemoveContainer" containerID="408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"
Nov 02 22:12:52.564709 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:52.564678    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1\": container with ID starting with 408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1 not found: ID does not exist" containerID="408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"
Nov 02 22:12:52.564818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.564717    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1"} err="failed to get container status \"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1\": rpc error: code = NotFound desc = could not find container \"408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1\": container with ID starting with 408af9a0175db876c061c99f0ee1cb769c2b3379af65b1c776cbd74511b1b0f1 not found: ID does not exist"
Nov 02 22:12:52.714606 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.714571    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:12:52.786274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.786237    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-027bdf-688dc4b5bc-9n89l"]
Nov 02 22:12:52.895743 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.895707    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:12:52.900141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.900107    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-027bdf-6dd899cff-pf4lm"]
Nov 02 22:12:52.976273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:52.976160    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:12:53.364423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.364389    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:12:53.521775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.521733    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.521804    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.521850    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.521896    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522001 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.521991    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522246 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522171    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522282    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522288    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:53.522365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522335    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522405    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\") pod \"4389b173-7d78-48c9-926f-065f5495ac7c\" (UID: \"4389b173-7d78-48c9-926f-065f5495ac7c\") "
Nov 02 22:12:53.522642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522584    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:53.523184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.522753    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:53.523184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.523017    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:53.523184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.523126    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.523184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.523149    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4389b173-7d78-48c9-926f-065f5495ac7c-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.523184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.523161    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.523517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.523426    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:53.525373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.525334    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:53.525745 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.525714    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "kube-api-access-tsm62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:53.525941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.525904    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:12:53.533556 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.533524    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 22:12:53.538225 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.538172    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 22:12:53.541057 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.541021    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo" pod "4389b173-7d78-48c9-926f-065f5495ac7c" (UID: "4389b173-7d78-48c9-926f-065f5495ac7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:12:53.547095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.546989    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 22:12:53.572721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.572625    5580 generic.go:356] "Generic (PLEG): container finished" podID="4389b173-7d78-48c9-926f-065f5495ac7c" containerID="32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a" exitCode=0
Nov 02 22:12:53.572970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.572940    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"
Nov 02 22:12:53.575358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.575211    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 22:12:53.575358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.575285    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerDied","Data":"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"}
Nov 02 22:12:53.575541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.575402    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h" event={"ID":"4389b173-7d78-48c9-926f-065f5495ac7c","Type":"ContainerDied","Data":"865e88847761d03561bed766aa79eea0ae25012f01ed37f55a921560cab26350"}
Nov 02 22:12:53.575541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.575441    5580 scope.go:122] "RemoveContainer" containerID="32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"
Nov 02 22:12:53.589845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.589817    5580 scope.go:122] "RemoveContainer" containerID="33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"
Nov 02 22:12:53.598426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.598390    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" path="/var/lib/kubelet/pods/3d213f6c-44ab-454a-adcd-90e7dea9a92e/volumes"
Nov 02 22:12:53.598983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.598946    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86701424-0da1-4aed-8f2d-cbe63230198e" path="/var/lib/kubelet/pods/86701424-0da1-4aed-8f2d-cbe63230198e/volumes"
Nov 02 22:12:53.599439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.599425    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" path="/var/lib/kubelet/pods/bfb4a535-9ea2-4ddb-a245-d374744a1c4a/volumes"
Nov 02 22:12:53.601588 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.601570    5580 scope.go:122] "RemoveContainer" containerID="32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"
Nov 02 22:12:53.602011 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:53.601987    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a\": container with ID starting with 32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a not found: ID does not exist" containerID="32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"
Nov 02 22:12:53.602100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.602023    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a"} err="failed to get container status \"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a\": rpc error: code = NotFound desc = could not find container \"32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a\": container with ID starting with 32d2f54a2c92d4ae769dedc1cb12461f06f945306549f3dfa8bcbdf26832983a not found: ID does not exist"
Nov 02 22:12:53.602100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.602048    5580 scope.go:122] "RemoveContainer" containerID="33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"
Nov 02 22:12:53.602341 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:53.602318    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76\": container with ID starting with 33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76 not found: ID does not exist" containerID="33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"
Nov 02 22:12:53.602395 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.602348    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76"} err="failed to get container status \"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76\": rpc error: code = NotFound desc = could not find container \"33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76\": container with ID starting with 33e2a4916e851ded7bc2e617af1a80f9ef590868a7764ffbe516a8142cc40d76 not found: ID does not exist"
Nov 02 22:12:53.624224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624185    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4389b173-7d78-48c9-926f-065f5495ac7c-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.624224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624217    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.624224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624227    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.624453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624236    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4389b173-7d78-48c9-926f-065f5495ac7c-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.624453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624269    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsm62\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-kube-api-access-tsm62\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.624453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.624280    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4389b173-7d78-48c9-926f-065f5495ac7c-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:53.983917 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.983877    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:12:53.988221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:53.988185    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-587629-openshift-default-75d8fc9cdf-qq96h"]
Nov 02 22:12:55.595437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:55.595405    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" path="/var/lib/kubelet/pods/4389b173-7d78-48c9-926f-065f5495ac7c/volumes"
Nov 02 22:12:56.312168 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:56.312128    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:12:57.977294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:57.977211    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" podUID="610477ee-b768-4bd3-8dfc-e4e062145374" containerName="payload-pre-processing" containerID="cri-o://93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229" gracePeriod=25
Nov 02 22:12:58.262203 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.262172    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:12:58.297276 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.297241    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\" (UniqueName: \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\") pod \"610477ee-b768-4bd3-8dfc-e4e062145374\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") "
Nov 02 22:12:58.297276 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.297279    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\") pod \"610477ee-b768-4bd3-8dfc-e4e062145374\" (UID: \"610477ee-b768-4bd3-8dfc-e4e062145374\") "
Nov 02 22:12:58.297738 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.297711    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume" pod "610477ee-b768-4bd3-8dfc-e4e062145374" (UID: "610477ee-b768-4bd3-8dfc-e4e062145374"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:58.297949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.297930    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/610477ee-b768-4bd3-8dfc-e4e062145374-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:58.299527 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.299496    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr" pod "610477ee-b768-4bd3-8dfc-e4e062145374" (UID: "610477ee-b768-4bd3-8dfc-e4e062145374"). InnerVolumeSpecName "kube-api-access-pm4pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:58.399742 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.399705    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm4pr\" (UniqueName: \"kubernetes.io/projected/610477ee-b768-4bd3-8dfc-e4e062145374-kube-api-access-pm4pr\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:12:58.618102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.618014    5580 generic.go:356] "Generic (PLEG): container finished" podID="610477ee-b768-4bd3-8dfc-e4e062145374" containerID="93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229" exitCode=0
Nov 02 22:12:58.618102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.618061    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"
Nov 02 22:12:58.618102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.618091    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" event={"ID":"610477ee-b768-4bd3-8dfc-e4e062145374","Type":"ContainerDied","Data":"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"}
Nov 02 22:12:58.618353 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.618147    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk" event={"ID":"610477ee-b768-4bd3-8dfc-e4e062145374","Type":"ContainerDied","Data":"9bc25fe7d0f3430e0a485200e438f760067b0e3f756de021d6efd42f702f5100"}
Nov 02 22:12:58.618353 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.618169    5580 scope.go:122] "RemoveContainer" containerID="93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"
Nov 02 22:12:58.631667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.631642    5580 scope.go:122] "RemoveContainer" containerID="93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"
Nov 02 22:12:58.632266 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:12:58.632239    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229\": container with ID starting with 93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229 not found: ID does not exist" containerID="93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"
Nov 02 22:12:58.632359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.632275    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229"} err="failed to get container status \"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229\": rpc error: code = NotFound desc = could not find container \"93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229\": container with ID starting with 93e58d22b88331e47bc7db3fba1d18a30288946b612d5b2ed321d0ca27293229 not found: ID does not exist"
Nov 02 22:12:58.782004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.781951    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:12:58.829852 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:58.829814    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2a9be7-5d58f5bb87-59gwk"]
Nov 02 22:12:59.594906 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:12:59.594872    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="610477ee-b768-4bd3-8dfc-e4e062145374" path="/var/lib/kubelet/pods/610477ee-b768-4bd3-8dfc-e4e062145374/volumes"
Nov 02 22:13:00.690317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:00.690279    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 22:13:00.690797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:00.690510    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" containerID="cri-o://b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8" gracePeriod=30
Nov 02 22:13:04.944408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:04.944364    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:13:04.944949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:04.944624    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" podUID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" containerName="maas-api" containerID="cri-o://9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df" gracePeriod=30
Nov 02 22:13:05.318933 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.318899    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:13:05.432468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.432428    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") pod \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") "
Nov 02 22:13:05.432697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.432670    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\") pod \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") "
Nov 02 22:13:05.432798 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.432776    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\" (UniqueName: \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\") pod \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") "
Nov 02 22:13:05.432869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.432849    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\") pod \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\" (UID: \"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c\") "
Nov 02 22:13:05.436101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.434267    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca" pod "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" (UID: "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:13:05.436651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.436606    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs" pod "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" (UID: "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:05.438636 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.438597    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls" pod "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" (UID: "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:05.446059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.446028    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c" pod "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" (UID: "cdf11c5b-fc58-4cdb-a515-f93fbeb4735c"). InnerVolumeSpecName "kube-api-access-g9j7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:05.540778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.539080    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9j7c\" (UniqueName: \"kubernetes.io/projected/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-kube-api-access-g9j7c\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:05.540778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.539122    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:05.540778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.539137    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:05.540778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.539151    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:05.670318 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.670285    5580 generic.go:356] "Generic (PLEG): container finished" podID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" containerID="9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df" exitCode=0
Nov 02 22:13:05.670519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.670504    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" event={"ID":"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c","Type":"ContainerDied","Data":"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"}
Nov 02 22:13:05.670660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.670648    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw" event={"ID":"cdf11c5b-fc58-4cdb-a515-f93fbeb4735c","Type":"ContainerDied","Data":"91a8b0a70a04c7480f7c40bcf2532694cb133abd9a38e53a6dc159c8f28d837e"}
Nov 02 22:13:05.670778 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.670767    5580 scope.go:122] "RemoveContainer" containerID="9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"
Nov 02 22:13:05.671033 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.671018    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"
Nov 02 22:13:05.725187 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.724440    5580 scope.go:122] "RemoveContainer" containerID="9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"
Nov 02 22:13:05.725187 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:05.725002    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df\": container with ID starting with 9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df not found: ID does not exist" containerID="9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"
Nov 02 22:13:05.725187 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.725047    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df"} err="failed to get container status \"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df\": rpc error: code = NotFound desc = could not find container \"9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df\": container with ID starting with 9e564fb2ba7e9b98370e4df6eedb1f9a1162c7d22847d6977f300c80751477df not found: ID does not exist"
Nov 02 22:13:05.845185 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.843119    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:13:05.847141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.846453    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-17435d-5bb548fdd4-whsfw"]
Nov 02 22:13:05.904984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.900229    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-5x54n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 02 22:13:05.904984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:05.900319    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 02 22:13:06.318982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.318220    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 22:13:06.361977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.361792    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362409    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362486    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362563    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362659    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362719    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.362997    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.363432    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.363597    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.364121    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.364319    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\") pod \"81913773-6f8a-4d3f-ad54-645788650e2a\" (UID: \"81913773-6f8a-4d3f-ad54-645788650e2a\") "
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.364928    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:13:06.365505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365264    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365725    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81913773-6f8a-4d3f-ad54-645788650e2a-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365758    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365773    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365789    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.365925    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "kube-api-access-r2dm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:06.366419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.366380    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:06.366773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.366427    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:06.366835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.366762    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:13:06.368256 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.368217    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy" pod "81913773-6f8a-4d3f-ad54-645788650e2a" (UID: "81913773-6f8a-4d3f-ad54-645788650e2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:06.467642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.467610    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2dm7\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-kube-api-access-r2dm7\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.467867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.467854    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81913773-6f8a-4d3f-ad54-645788650e2a-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.467934 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.467927    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81913773-6f8a-4d3f-ad54-645788650e2a-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.468084 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.468070    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.468165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.468157    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81913773-6f8a-4d3f-ad54-645788650e2a-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:06.716559 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.716513    5580 generic.go:356] "Generic (PLEG): container finished" podID="81913773-6f8a-4d3f-ad54-645788650e2a" containerID="b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8" exitCode=0
Nov 02 22:13:06.716888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.716860    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" event={"ID":"81913773-6f8a-4d3f-ad54-645788650e2a","Type":"ContainerDied","Data":"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"}
Nov 02 22:13:06.717123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.717103    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n" event={"ID":"81913773-6f8a-4d3f-ad54-645788650e2a","Type":"ContainerDied","Data":"84a3f784295ada49a9ab435fb1119988123cdc8cfc9a44fe969452e659af0700"}
Nov 02 22:13:06.717249 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.717235    5580 scope.go:122] "RemoveContainer" containerID="b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"
Nov 02 22:13:06.717523 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.717504    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"
Nov 02 22:13:06.772702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.772224    5580 scope.go:122] "RemoveContainer" containerID="b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"
Nov 02 22:13:06.773391 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:06.773343    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8\": container with ID starting with b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8 not found: ID does not exist" containerID="b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"
Nov 02 22:13:06.773558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.773421    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8"} err="failed to get container status \"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8\": rpc error: code = NotFound desc = could not find container \"b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8\": container with ID starting with b0cd891efda806b71cd84831f62b466b7c96f96c9a844a95ca34d4066857c2a8 not found: ID does not exist"
Nov 02 22:13:06.929977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.929667    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 22:13:06.935651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:06.935605    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-5x54n"]
Nov 02 22:13:07.638980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:07.638831    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" path="/var/lib/kubelet/pods/81913773-6f8a-4d3f-ad54-645788650e2a/volumes"
Nov 02 22:13:07.642980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:07.639625    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" path="/var/lib/kubelet/pods/cdf11c5b-fc58-4cdb-a515-f93fbeb4735c/volumes"
Nov 02 22:13:08.332983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.328629    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:13:08.376128 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.376069    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:13:08.562132 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.562082    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc" podUID="9b51a5e6-f2f4-4349-bf3e-47774cc29115" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:13:08.597350 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.597237    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" containerID="cri-o://20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc" gracePeriod=45
Nov 02 22:13:08.632503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.632444    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main" containerID="cri-o://64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19" gracePeriod=45
Nov 02 22:13:08.633993 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.633936    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" containerID="cri-o://038ccdfacaead892edc504dd6e4b6f83c954cc192b72ee5a9ad00df00d972060" gracePeriod=45
Nov 02 22:13:08.645679 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.645641    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" containerID="cri-o://2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00" gracePeriod=45
Nov 02 22:13:08.653238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.652453    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": EOF"
Nov 02 22:13:08.673227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.673184    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": EOF"
Nov 02 22:13:08.675686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.675306    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 22:13:08.949148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.948892    5580 generic.go:356] "Generic (PLEG): container finished" podID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerID="038ccdfacaead892edc504dd6e4b6f83c954cc192b72ee5a9ad00df00d972060" exitCode=0
Nov 02 22:13:08.949148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:08.949030    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" event={"ID":"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3","Type":"ContainerDied","Data":"038ccdfacaead892edc504dd6e4b6f83c954cc192b72ee5a9ad00df00d972060"}
Nov 02 22:13:09.242531 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.242499    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 22:13:09.247725 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.247692    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 22:13:09.324554 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.324525    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 22:13:09.325015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.324995    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.325154 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.325142    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.325291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.325280    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.325662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.325502    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.325662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.325534    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.325816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.325803    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.326211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.326010    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.326346 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.326301    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\" (UniqueName: \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.326489 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.326477    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\" (UniqueName: \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.326578 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.326568    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.326715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.326705    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\" (UniqueName: \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.327502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.327488    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.327827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.327816    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.328461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328040    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\") pod \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\" (UID: \"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3\") "
Nov 02 22:13:09.328461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328085    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\") pod \"048eb4a2-d770-4485-aa56-fd39eedae514\" (UID: \"048eb4a2-d770-4485-aa56-fd39eedae514\") "
Nov 02 22:13:09.328733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328714    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.328805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328738    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.328805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328752    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.328805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.327770    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.328805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328085    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.328805 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.328237    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.329772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.329632    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "kube-api-access-6sg6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:09.329772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.329737    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:09.330110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.330086    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.330384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.330342    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:09.332760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.332739    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx" pod "048eb4a2-d770-4485-aa56-fd39eedae514" (UID: "048eb4a2-d770-4485-aa56-fd39eedae514"). InnerVolumeSpecName "kube-api-access-bj4kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:09.334453 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.334426    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm" pod "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" (UID: "1d56c8c7-0d51-4029-84a1-f8fe0ae478e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.340872 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.340814    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 22:13:09.429428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429390    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.429582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429443    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\" (UniqueName: \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.429582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429503    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.429582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429525    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429565    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.429740 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429608    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.429740 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429631    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429740 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429678    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429740 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429693    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429863 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429755    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429863 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429775    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\" (UniqueName: \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\") pod \"dbf12759-76ee-4a9f-b430-1431d6d211d9\" (UID: \"dbf12759-76ee-4a9f-b430-1431d6d211d9\") "
Nov 02 22:13:09.429863 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.429843    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\") pod \"3f49eecd-1382-43d4-a04a-1d0765533024\" (UID: \"3f49eecd-1382-43d4-a04a-1d0765533024\") "
Nov 02 22:13:09.430290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430264    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/048eb4a2-d770-4485-aa56-fd39eedae514-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430280    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430289    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430298    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430306    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/048eb4a2-d770-4485-aa56-fd39eedae514-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430314    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430323    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sg6v\" (UniqueName: \"kubernetes.io/projected/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-kube-api-access-6sg6v\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430332    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj4kx\" (UniqueName: \"kubernetes.io/projected/048eb4a2-d770-4485-aa56-fd39eedae514-kube-api-access-bj4kx\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.430405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.430341    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.431720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431268    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.431720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431279    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.431720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431428    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.431720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431461    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.431720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431666    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.432090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.431772    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.434160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.434127    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.434401 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.434371    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "kube-api-access-rnwgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:09.434733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.434707    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:09.434806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.434716    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "kube-api-access-kqwdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:09.434994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.434923    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs" pod "dbf12759-76ee-4a9f-b430-1431d6d211d9" (UID: "dbf12759-76ee-4a9f-b430-1431d6d211d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:09.435161 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.435143    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm" pod "3f49eecd-1382-43d4-a04a-1d0765533024" (UID: "3f49eecd-1382-43d4-a04a-1d0765533024"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:09.531968 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.531923    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.531980    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqwdl\" (UniqueName: \"kubernetes.io/projected/3f49eecd-1382-43d4-a04a-1d0765533024-kube-api-access-kqwdl\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532000    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532016    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532031    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f49eecd-1382-43d4-a04a-1d0765533024-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532044    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532057    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532070    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532082    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf12759-76ee-4a9f-b430-1431d6d211d9-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532094    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf12759-76ee-4a9f-b430-1431d6d211d9-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532106    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnwgv\" (UniqueName: \"kubernetes.io/projected/dbf12759-76ee-4a9f-b430-1431d6d211d9-kube-api-access-rnwgv\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.532133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.532119    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f49eecd-1382-43d4-a04a-1d0765533024-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:09.970595 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.970556    5580 generic.go:356] "Generic (PLEG): container finished" podID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerID="64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19" exitCode=0
Nov 02 22:13:09.971149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.970643    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"
Nov 02 22:13:09.971149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.970646    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" event={"ID":"dbf12759-76ee-4a9f-b430-1431d6d211d9","Type":"ContainerDied","Data":"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"}
Nov 02 22:13:09.971149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.970704    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm" event={"ID":"dbf12759-76ee-4a9f-b430-1431d6d211d9","Type":"ContainerDied","Data":"6309b53a9eb2d42b7ddf0bc1ab6b58832cc8ce5d462bb6b7e1a59ca9a9caa3e2"}
Nov 02 22:13:09.971149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.970726    5580 scope.go:122] "RemoveContainer" containerID="64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"
Nov 02 22:13:09.973789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.973070    5580 generic.go:356] "Generic (PLEG): container finished" podID="048eb4a2-d770-4485-aa56-fd39eedae514" containerID="2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00" exitCode=0
Nov 02 22:13:09.973789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.973193    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" event={"ID":"048eb4a2-d770-4485-aa56-fd39eedae514","Type":"ContainerDied","Data":"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"}
Nov 02 22:13:09.973789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.973226    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n" event={"ID":"048eb4a2-d770-4485-aa56-fd39eedae514","Type":"ContainerDied","Data":"d8c922fd6902db579dc11ab0d31fd71363e62d2f844edc2ef8f1268b214d6f62"}
Nov 02 22:13:09.973789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.973311    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"
Nov 02 22:13:09.975281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.975262    5580 generic.go:356] "Generic (PLEG): container finished" podID="3f49eecd-1382-43d4-a04a-1d0765533024" containerID="20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc" exitCode=0
Nov 02 22:13:09.975458 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.975443    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" event={"ID":"3f49eecd-1382-43d4-a04a-1d0765533024","Type":"ContainerDied","Data":"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"}
Nov 02 22:13:09.975543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.975531    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" event={"ID":"3f49eecd-1382-43d4-a04a-1d0765533024","Type":"ContainerDied","Data":"2214c2cf5ccaf7ddcce75a0775e1868f1a2a7ea3fac2275ccf29ddb63bf167df"}
Nov 02 22:13:09.990146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:09.990097    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"
Nov 02 22:13:10.005000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.004934    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq" event={"ID":"1d56c8c7-0d51-4029-84a1-f8fe0ae478e3","Type":"ContainerDied","Data":"c625158dddb4b779fcc296ec778dfd42b25002ea8b3964c63de7e56ffc9417aa"}
Nov 02 22:13:10.005171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.005015    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"
Nov 02 22:13:10.010471 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.010448    5580 scope.go:122] "RemoveContainer" containerID="64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"
Nov 02 22:13:10.010797 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:10.010775    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19\": container with ID starting with 64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19 not found: ID does not exist" containerID="64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"
Nov 02 22:13:10.010836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.010810    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19"} err="failed to get container status \"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19\": rpc error: code = NotFound desc = could not find container \"64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19\": container with ID starting with 64f721bd8f778781c0a54ffffff845e33f7afa9a34392649bd8544d6745fbc19 not found: ID does not exist"
Nov 02 22:13:10.010836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.010830    5580 scope.go:122] "RemoveContainer" containerID="2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"
Nov 02 22:13:10.019967 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.019936    5580 scope.go:122] "RemoveContainer" containerID="2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"
Nov 02 22:13:10.020308 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:10.020291    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00\": container with ID starting with 2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00 not found: ID does not exist" containerID="2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"
Nov 02 22:13:10.020375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.020318    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00"} err="failed to get container status \"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00\": rpc error: code = NotFound desc = could not find container \"2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00\": container with ID starting with 2ca5d3466802ca212f6c950a49464f973a5b548b8d55c2775b860fe0840bde00 not found: ID does not exist"
Nov 02 22:13:10.020375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.020335    5580 scope.go:122] "RemoveContainer" containerID="20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"
Nov 02 22:13:10.029878 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.029852    5580 scope.go:122] "RemoveContainer" containerID="20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"
Nov 02 22:13:10.030255 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:10.030230    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc\": container with ID starting with 20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc not found: ID does not exist" containerID="20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"
Nov 02 22:13:10.030355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.030263    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc"} err="failed to get container status \"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc\": rpc error: code = NotFound desc = could not find container \"20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc\": container with ID starting with 20a637b83553a33382c31a8533153bd5f98d1116a2651c087b48636df080f4bc not found: ID does not exist"
Nov 02 22:13:10.030355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.030283    5580 scope.go:122] "RemoveContainer" containerID="038ccdfacaead892edc504dd6e4b6f83c954cc192b72ee5a9ad00df00d972060"
Nov 02 22:13:10.132524 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.132486    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 22:13:10.146161 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.146112    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:13:10.202189 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.202132    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-2-w0-f48c4d-kserve-68c8b6fc7-4k79n"]
Nov 02 22:13:10.248695 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.248611    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 22:13:10.253971 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.253928    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/simulator-w0-f48c4d-kserve-bf49d9bb-5nwbm"]
Nov 02 22:13:10.363771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.363725    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 22:13:10.367412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.367384    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/distinct-w0-f48c4d-kserve-54c946f7fd-b8hqq"]
Nov 02 22:13:10.475056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.475020    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 22:13:10.480277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:10.480241    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f48c4d/premium-w0-f48c4d-kserve-67dc4b45f-scpxz"]
Nov 02 22:13:11.596045 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:11.596010    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" path="/var/lib/kubelet/pods/048eb4a2-d770-4485-aa56-fd39eedae514/volumes"
Nov 02 22:13:11.596601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:11.596583    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" path="/var/lib/kubelet/pods/1d56c8c7-0d51-4029-84a1-f8fe0ae478e3/volumes"
Nov 02 22:13:11.597167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:11.597148    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" path="/var/lib/kubelet/pods/3f49eecd-1382-43d4-a04a-1d0765533024/volumes"
Nov 02 22:13:11.598265 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:11.598243    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" path="/var/lib/kubelet/pods/dbf12759-76ee-4a9f-b430-1431d6d211d9/volumes"
Nov 02 22:13:15.051429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.051394    5580 generic.go:356] "Generic (PLEG): container finished" podID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerID="4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a" exitCode=0
Nov 02 22:13:15.051873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.051477    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerDied","Data":"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"}
Nov 02 22:13:15.052073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.052055    5580 scope.go:122] "RemoveContainer" containerID="4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"
Nov 02 22:13:15.053799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.053769    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:15.053862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.053845    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:15.053898 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.053874    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:15.695110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.695076    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 22:13:15.695348 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:15.695321    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" containerID="cri-o://36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09" gracePeriod=30
Nov 02 22:13:16.060844 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:16.060759    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerStarted","Data":"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"}
Nov 02 22:13:16.207983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:16.193396    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:13:16.207983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:16.202081    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:16.207983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:16.202144    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:17.197982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:17.197433    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:17.197982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:17.197506    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:18.196188 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:18.196148    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:18.196481 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:18.196452    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:18.966278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:18.966226    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:13:19.200425 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:19.200381    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:19.200735 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:19.200704    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:20.194422 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:20.194377    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:20.195153 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:20.195112    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:21.034652 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.034625    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122745    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122810    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122849    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122892    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122913    5580 generic.go:356] "Generic (PLEG): container finished" podID="98a84e52-4615-432d-9bb5-ed52d0203507" containerID="36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09" exitCode=0
Nov 02 22:13:21.122994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.122938    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123018    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" event={"ID":"98a84e52-4615-432d-9bb5-ed52d0203507","Type":"ContainerDied","Data":"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"}
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123074    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" event={"ID":"98a84e52-4615-432d-9bb5-ed52d0203507","Type":"ContainerDied","Data":"88d5938e6269a5ba24bb6e10b48fa0024e0f68310d6c4dd6a84e1601a0b10133"}
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123079    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123105    5580 scope.go:122] "RemoveContainer" containerID="36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123165    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123225    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123321    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.123438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123357    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.123823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123459    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\") pod \"98a84e52-4615-432d-9bb5-ed52d0203507\" (UID: \"98a84e52-4615-432d-9bb5-ed52d0203507\") "
Nov 02 22:13:21.123823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123566    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:21.123929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.123837    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:21.124282 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.124254    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.124409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.124285    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.124409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.124302    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.124553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.124529    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:13:21.124618 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.124602    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:21.125951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.125911    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:13:21.126079 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.126041    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:13:21.126812 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.126787    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:21.127590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.127565    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75" pod "98a84e52-4615-432d-9bb5-ed52d0203507" (UID: "98a84e52-4615-432d-9bb5-ed52d0203507"). InnerVolumeSpecName "kube-api-access-6qk75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:21.165835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.165798    5580 scope.go:122] "RemoveContainer" containerID="36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"
Nov 02 22:13:21.166269 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:21.166241    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09\": container with ID starting with 36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09 not found: ID does not exist" containerID="36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"
Nov 02 22:13:21.166365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.166282    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09"} err="failed to get container status \"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09\": rpc error: code = NotFound desc = could not find container \"36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09\": container with ID starting with 36b9032c8c6cc4946922b58778a251950e30a350eb99caeab678406465f6bd09 not found: ID does not exist"
Nov 02 22:13:21.197387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.194383    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 02 22:13:21.197387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.194470    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 02 22:13:21.226068 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226027    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.226298 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226285    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a84e52-4615-432d-9bb5-ed52d0203507-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.226359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226352    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.226414 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226407    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a84e52-4615-432d-9bb5-ed52d0203507-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.226473 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226465    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a84e52-4615-432d-9bb5-ed52d0203507-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.226526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.226519    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qk75\" (UniqueName: \"kubernetes.io/projected/98a84e52-4615-432d-9bb5-ed52d0203507-kube-api-access-6qk75\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:21.544561 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.544524    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 22:13:21.547650 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.547615    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2"]
Nov 02 22:13:21.594824 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.594785    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" path="/var/lib/kubelet/pods/98a84e52-4615-432d-9bb5-ed52d0203507/volumes"
Nov 02 22:13:21.901209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.901112    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-xkdf2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:13:21.901209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:21.901176    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-xkdf2" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:13:22.197365 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:22.197332    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:13:22.198472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:22.198452    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:13:26.311579 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:26.311537    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:13:27.692119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:27.692083    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:13:30.706292 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706250    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:13:30.706878 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706861    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706878    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706892    5580 cpu_manager.go:414] "containerMap: removing container" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706897    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706910    5580 cpu_manager.go:414] "containerMap: removing container" podUID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" containerName="maas-api"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706915    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" containerName="maas-api"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706922    5580 cpu_manager.go:414] "containerMap: removing container" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy"
Nov 02 22:13:30.706926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706927    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706934    5580 cpu_manager.go:414] "containerMap: removing container" podUID="92249a10-e668-4fcc-9bbf-8770552b2563" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706939    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92249a10-e668-4fcc-9bbf-8770552b2563" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706948    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706965    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706973    5580 cpu_manager.go:414] "containerMap: removing container" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706978    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706983    5580 cpu_manager.go:414] "containerMap: removing container" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706987    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.706997    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707002    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707011    5580 cpu_manager.go:414] "containerMap: removing container" podUID="86701424-0da1-4aed-8f2d-cbe63230198e" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707015    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86701424-0da1-4aed-8f2d-cbe63230198e" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707022    5580 cpu_manager.go:414] "containerMap: removing container" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707026    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707033    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1f474e96-2212-474d-b7e4-b8813cdc398e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707038    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f474e96-2212-474d-b7e4-b8813cdc398e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707045    5580 cpu_manager.go:414] "containerMap: removing container" podUID="610477ee-b768-4bd3-8dfc-e4e062145374" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707050    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="610477ee-b768-4bd3-8dfc-e4e062145374" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707056    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707062    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707080    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707085    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707091    5580 cpu_manager.go:414] "containerMap: removing container" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707095    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707107    5580 cpu_manager.go:414] "containerMap: removing container" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707111    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707117    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d60eb4a7-af94-4e56-89d7-bf3202287774" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707122    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d60eb4a7-af94-4e56-89d7-bf3202287774" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707129    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707134    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707235    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c8dc48b-3c3a-4def-abf8-1638d8e47a4e" containerName="istio-proxy"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707243    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="86701424-0da1-4aed-8f2d-cbe63230198e" containerName="payload-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707250    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbf12759-76ee-4a9f-b430-1431d6d211d9" containerName="main"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707255    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f474e96-2212-474d-b7e4-b8813cdc398e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707259    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="610477ee-b768-4bd3-8dfc-e4e062145374" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707264    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d213f6c-44ab-454a-adcd-90e7dea9a92e" containerName="payload-pre-processing"
Nov 02 22:13:30.707258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707271    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707280    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f49eecd-1382-43d4-a04a-1d0765533024" containerName="main"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707286    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="048eb4a2-d770-4485-aa56-fd39eedae514" containerName="main"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707292    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e14dba8-4d31-40b5-b273-67ca8d96ddb9" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707299    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d60eb4a7-af94-4e56-89d7-bf3202287774" containerName="payload-processing"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707305    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9a3988e-fac7-4132-93b5-6f97f7ac7c32" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707311    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707316    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="98a84e52-4615-432d-9bb5-ed52d0203507" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707321    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdf11c5b-fc58-4cdb-a515-f93fbeb4735c" containerName="maas-api"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707326    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="04d6874f-bb2c-4142-9793-481d0b108b6b" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707333    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="92249a10-e668-4fcc-9bbf-8770552b2563" containerName="payload-processing"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707338    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d56c8c7-0d51-4029-84a1-f8fe0ae478e3" containerName="main"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707344    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfb4a535-9ea2-4ddb-a245-d374744a1c4a" containerName="istio-proxy"
Nov 02 22:13:30.708412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.707349    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="81913773-6f8a-4d3f-ad54-645788650e2a" containerName="istio-proxy"
Nov 02 22:13:30.711009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.710984    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771378    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771430    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771458    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771479    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771504    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771571    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74nnj\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771598    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771615    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.771831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.771698    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.781682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.781640    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:13:30.782731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.782698    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.782731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.782730    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4389b173-7d78-48c9-926f-065f5495ac7c" containerName="istio-proxy"
Nov 02 22:13:30.788599 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.788571    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.816248 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.816210    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:13:30.829333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.829298    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:13:30.872599 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872543    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.872818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872684    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74nnj\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.872818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872731    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.872818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872759    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.872818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872790    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.873056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872856    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.873056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872904    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.873056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.872940    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.873056 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.873019    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.873274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.873049    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.873274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.873060    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.873386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.873357    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.874439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.874415    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.874780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.874759    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzfmq\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.874904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.874884    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.875023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.874936    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.875023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.874985    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.875140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.875030    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.875140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.875057    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.875140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.875101    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.875140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.875127    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.875913 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.875886    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.876163 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.876063    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.882248 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.878540    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.884397 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.884365    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.929363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.929329    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74nnj\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.929863 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.929844    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-k6fj4\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:30.976408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976301    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976384    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976431    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976462    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976512    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976562    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzfmq\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976610    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976718    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976803    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976855    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976860    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.976938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.976920    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.977253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.977142    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.977253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.977222    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.979115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.979093    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:30.979238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:30.979192    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:31.033783 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.033749    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzfmq\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:31.033964 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.033935    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-87q86\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:31.061221 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.061185    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:31.104522 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.104482    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:31.490165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.490126    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:13:31.530407 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:31.530369    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5951710_8983_438c_ae0a_a7749eb1cd32.slice/crio-40ed742bc50ea2cdbc67aeed795680280120d632672651518f24e51091ce7975 WatchSource:0}: Error finding container 40ed742bc50ea2cdbc67aeed795680280120d632672651518f24e51091ce7975: Status 404 returned error can't find the container with id 40ed742bc50ea2cdbc67aeed795680280120d632672651518f24e51091ce7975
Nov 02 22:13:31.533241 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.533196    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.533350 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.533296    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.533350 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.533338    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.551700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.551645    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:13:31.566582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.566544    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.566709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.566619    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.566709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.566651    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:31.788075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.784969    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:13:31.788075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:31.785234    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" podUID="ee30fd55-f019-40aa-8326-c2aff76f2951" containerName="maas-api" containerID="cri-o://fd23a57cd7baf2b379513149eddf874d15c0b140ae6cd615066ae100efdc2d98" gracePeriod=30
Nov 02 22:13:32.224553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.224504    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" event={"ID":"74d46a2f-20e1-4b5b-b169-e8610095770f","Type":"ContainerStarted","Data":"ac34eeec4ff75bef7b5737fdf4b0c007123d5d5b790b9d0ebfa4028645634870"}
Nov 02 22:13:32.224823 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.224588    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" event={"ID":"74d46a2f-20e1-4b5b-b169-e8610095770f","Type":"ContainerStarted","Data":"56db105449ef1912a748ca41286770dbe618f81111e6fe273b1af18499e8ca96"}
Nov 02 22:13:32.228095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.228058    5580 generic.go:356] "Generic (PLEG): container finished" podID="ee30fd55-f019-40aa-8326-c2aff76f2951" containerID="fd23a57cd7baf2b379513149eddf874d15c0b140ae6cd615066ae100efdc2d98" exitCode=0
Nov 02 22:13:32.228207 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.228113    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" event={"ID":"ee30fd55-f019-40aa-8326-c2aff76f2951","Type":"ContainerDied","Data":"fd23a57cd7baf2b379513149eddf874d15c0b140ae6cd615066ae100efdc2d98"}
Nov 02 22:13:32.230948 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.230900    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" event={"ID":"a5951710-8983-438c-ae0a-a7749eb1cd32","Type":"ContainerStarted","Data":"e0b0d91945c504e9f104cbfe4b0e4d916846ad6dc4244e0d56c721a9dd0b478a"}
Nov 02 22:13:32.231039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.230950    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" event={"ID":"a5951710-8983-438c-ae0a-a7749eb1cd32","Type":"ContainerStarted","Data":"40ed742bc50ea2cdbc67aeed795680280120d632672651518f24e51091ce7975"}
Nov 02 22:13:32.267116 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.267079    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:13:32.327688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.327636    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\") pod \"ee30fd55-f019-40aa-8326-c2aff76f2951\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") "
Nov 02 22:13:32.327869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.327742    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") pod \"ee30fd55-f019-40aa-8326-c2aff76f2951\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") "
Nov 02 22:13:32.327869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.327822    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\") pod \"ee30fd55-f019-40aa-8326-c2aff76f2951\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") "
Nov 02 22:13:32.328086 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.327948    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\" (UniqueName: \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\") pod \"ee30fd55-f019-40aa-8326-c2aff76f2951\" (UID: \"ee30fd55-f019-40aa-8326-c2aff76f2951\") "
Nov 02 22:13:32.330255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.330220    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca" pod "ee30fd55-f019-40aa-8326-c2aff76f2951" (UID: "ee30fd55-f019-40aa-8326-c2aff76f2951"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:13:32.333073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.333040    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs" pod "ee30fd55-f019-40aa-8326-c2aff76f2951" (UID: "ee30fd55-f019-40aa-8326-c2aff76f2951"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:32.333553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.333525    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc" pod "ee30fd55-f019-40aa-8326-c2aff76f2951" (UID: "ee30fd55-f019-40aa-8326-c2aff76f2951"). InnerVolumeSpecName "kube-api-access-6r2cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:32.333820 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.333796    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls" pod "ee30fd55-f019-40aa-8326-c2aff76f2951" (UID: "ee30fd55-f019-40aa-8326-c2aff76f2951"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:13:32.350914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.346164    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podStartSLOduration=2.346144674 podStartE2EDuration="2.346144674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:32.342331434 +0000 UTC m=+2310.858827212" watchObservedRunningTime="2026-11-02 22:13:32.346144674 +0000 UTC m=+2310.862640449"
Nov 02 22:13:32.431755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.430998    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:32.431755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.431040    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee30fd55-f019-40aa-8326-c2aff76f2951-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:32.431755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.431057    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r2cc\" (UniqueName: \"kubernetes.io/projected/ee30fd55-f019-40aa-8326-c2aff76f2951-kube-api-access-6r2cc\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:32.431755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.431073    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee30fd55-f019-40aa-8326-c2aff76f2951-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:32.632035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.631711    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podStartSLOduration=2.631684711 podStartE2EDuration="2.631684711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:32.599476425 +0000 UTC m=+2311.115972200" watchObservedRunningTime="2026-11-02 22:13:32.631684711 +0000 UTC m=+2311.148180486"
Nov 02 22:13:32.708254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:32.708037    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" podUID="68f9ec66-73df-47c7-9a17-8778899610b7" containerName="payload-pre-processing" containerID="cri-o://e08c8d0bad4f3687ca9c3b8ccd285a7bb55f4445f02c69519cdc001369b642f9" gracePeriod=25
Nov 02 22:13:33.063666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.063259    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:33.063666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.063368    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:33.063666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.063406    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:33.117308 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.111046    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:33.142300 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.142250    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:33.142591 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.142549    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:33.269000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.259701    5580 generic.go:356] "Generic (PLEG): container finished" podID="68f9ec66-73df-47c7-9a17-8778899610b7" containerID="e08c8d0bad4f3687ca9c3b8ccd285a7bb55f4445f02c69519cdc001369b642f9" exitCode=0
Nov 02 22:13:33.269000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.259747    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" event={"ID":"68f9ec66-73df-47c7-9a17-8778899610b7","Type":"ContainerDied","Data":"e08c8d0bad4f3687ca9c3b8ccd285a7bb55f4445f02c69519cdc001369b642f9"}
Nov 02 22:13:33.269000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.261772    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"
Nov 02 22:13:33.269000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.261807    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt" event={"ID":"ee30fd55-f019-40aa-8326-c2aff76f2951","Type":"ContainerDied","Data":"9be5da8706149e3bf294bfcd509ebb1b7c70a8c93abfc3243ac92db8a78252a6"}
Nov 02 22:13:33.269000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.261857    5580 scope.go:122] "RemoveContainer" containerID="fd23a57cd7baf2b379513149eddf874d15c0b140ae6cd615066ae100efdc2d98"
Nov 02 22:13:33.343438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.343352    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:13:33.360306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.360261    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\") pod \"68f9ec66-73df-47c7-9a17-8778899610b7\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") "
Nov 02 22:13:33.361091 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.360650    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\" (UniqueName: \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\") pod \"68f9ec66-73df-47c7-9a17-8778899610b7\" (UID: \"68f9ec66-73df-47c7-9a17-8778899610b7\") "
Nov 02 22:13:33.361091 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.360663    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume" pod "68f9ec66-73df-47c7-9a17-8778899610b7" (UID: "68f9ec66-73df-47c7-9a17-8778899610b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:13:33.361717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.361469    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68f9ec66-73df-47c7-9a17-8778899610b7-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:33.362951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.362917    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s" pod "68f9ec66-73df-47c7-9a17-8778899610b7" (UID: "68f9ec66-73df-47c7-9a17-8778899610b7"). InnerVolumeSpecName "kube-api-access-6n58s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:13:33.442420 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.442380    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:13:33.445775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.445744    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-bdc145-778d865748-cxhvt"]
Nov 02 22:13:33.463315 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.463236    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n58s\" (UniqueName: \"kubernetes.io/projected/68f9ec66-73df-47c7-9a17-8778899610b7-kube-api-access-6n58s\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:13:33.641714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:33.641679    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee30fd55-f019-40aa-8326-c2aff76f2951" path="/var/lib/kubelet/pods/ee30fd55-f019-40aa-8326-c2aff76f2951/volumes"
Nov 02 22:13:34.061746 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.061713    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:34.062064 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.062033    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:34.120070 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.120021    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:34.120704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.120664    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:34.270663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.270095    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"
Nov 02 22:13:34.270663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.270131    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd" event={"ID":"68f9ec66-73df-47c7-9a17-8778899610b7","Type":"ContainerDied","Data":"4167d78beea53cdfe2b8815f2f7a929af7fd55ea70b33295453c5ed045cd64ca"}
Nov 02 22:13:34.270663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.270203    5580 scope.go:122] "RemoveContainer" containerID="e08c8d0bad4f3687ca9c3b8ccd285a7bb55f4445f02c69519cdc001369b642f9"
Nov 02 22:13:34.463230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.463185    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:13:34.465903 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:34.465861    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-caac72-5d76f49855-dssnd"]
Nov 02 22:13:35.028982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.028765    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.029804    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ee30fd55-f019-40aa-8326-c2aff76f2951" containerName="maas-api"
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.029832    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee30fd55-f019-40aa-8326-c2aff76f2951" containerName="maas-api"
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.029849    5580 cpu_manager.go:414] "containerMap: removing container" podUID="68f9ec66-73df-47c7-9a17-8778899610b7" containerName="payload-pre-processing"
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.029857    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68f9ec66-73df-47c7-9a17-8778899610b7" containerName="payload-pre-processing"
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.030028    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee30fd55-f019-40aa-8326-c2aff76f2951" containerName="maas-api"
Nov 02 22:13:35.030985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.030042    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="68f9ec66-73df-47c7-9a17-8778899610b7" containerName="payload-pre-processing"
Nov 02 22:13:35.046833 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.043615    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.069581 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.069543    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:35.069708 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.069635    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.151977    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhbp\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162117    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162239    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162389    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162419    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162498    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162529    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162590    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162626    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.155733    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.157375    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:35.163013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.162933    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:35.169033 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.168436    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-af9e7c-openshift-default-dockercfg-lx2h9\""
Nov 02 22:13:35.263899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.263857    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhbp\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.264529 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.264503    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.267179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.264758    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.267461 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.267438    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.268071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.267885    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268618    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268658    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268713    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268750    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.267895    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268990    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.268078    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.269766 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.269714    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.270656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.270032    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.271160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.271114    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.271483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.271464    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.349730 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.349619    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.354782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.354756    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhbp\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\") pod \"e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") " pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.402643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.402316    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:35.643722 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.640631    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68f9ec66-73df-47c7-9a17-8778899610b7" path="/var/lib/kubelet/pods/68f9ec66-73df-47c7-9a17-8778899610b7/volumes"
Nov 02 22:13:35.935024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.934981    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:13:35.938155 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:35.938118    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bf8038_6dfd_4c72_85ac_1a222b061fa4.slice/crio-283fa8bf73a020c690af1341044ba95f03570334cca9012911050677287175d8 WatchSource:0}: Error finding container 283fa8bf73a020c690af1341044ba95f03570334cca9012911050677287175d8: Status 404 returned error can't find the container with id 283fa8bf73a020c690af1341044ba95f03570334cca9012911050677287175d8
Nov 02 22:13:35.941203 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.941172    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:35.941289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.941243    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:35.941289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:35.941273    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:36.062502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.062142    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:36.062502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.062191    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:36.110996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.108911    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:36.110996 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.108997    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:36.354368 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.354280    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerStarted","Data":"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"}
Nov 02 22:13:36.354368 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.354334    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerStarted","Data":"283fa8bf73a020c690af1341044ba95f03570334cca9012911050677287175d8"}
Nov 02 22:13:36.403993 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:36.403058    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:37.062557 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.062515    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:37.062775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.062583    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:37.106980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.106281    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:37.106980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.106346    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:37.425947 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.425506    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:37.545832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:37.545772    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podStartSLOduration=3.545755821 podStartE2EDuration="3.545755821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:36.494377796 +0000 UTC m=+2315.010873582" watchObservedRunningTime="2026-11-02 22:13:37.545755821 +0000 UTC m=+2316.062251594"
Nov 02 22:13:38.062408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.062370    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:38.062611 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.062428    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:38.105844 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.105804    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:38.106179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.106149    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:38.367314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.367222    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:38.368257 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:38.368234    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:13:39.062017 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:39.061976    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-k6fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Nov 02 22:13:39.062426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:39.062044    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Nov 02 22:13:39.105434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:39.105388    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-87q86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Nov 02 22:13:39.105614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:39.105463    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Nov 02 22:13:40.717217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.717184    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:40.718194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.718154    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:40.719026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.719006    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:13:40.812824 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.812764    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:40.813484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.813459    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:40.814555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:40.814531    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:13:43.110137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.110079    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:13:43.129339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.129303    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.164634 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.164600    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:13:43.210097 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.210050    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-af9e7c\""
Nov 02 22:13:43.210359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.210113    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-af9e7c-dockercfg-zhqhv\""
Nov 02 22:13:43.272243 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.272206    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:13:43.276660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.276624    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.292143 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.292110    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.292314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.292295    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6jpq\" (UniqueName: \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.333649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.333614    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:13:43.393133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.393024    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjf2c\" (UniqueName: \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.393133 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.393099    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6jpq\" (UniqueName: \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.393393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.393276    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.393393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.393330    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.393982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.393947    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.450547 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.450508    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6jpq\" (UniqueName: \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\") pod \"payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.469402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.469369    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:43.494474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.494435    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.494651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.494522    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjf2c\" (UniqueName: \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.495321 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.495290    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.557855 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.557760    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjf2c\" (UniqueName: \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\") pod \"payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.661865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.616387    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:43.769491 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.769459    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:13:43.775496 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.775468    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.818792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.818757    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.818801    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.818933    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819136 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819116    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819174 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819153    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfg7s\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819271 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819254    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819346 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819323    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819410    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.819506 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.819491    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.834769 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.834738    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d1a155-openshift-default-dockercfg-xxcwh\""
Nov 02 22:13:43.838051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.838019    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:13:43.879562 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.879534    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:13:43.881365 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:43.881334    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12236a2c_5fca_46cd_aaa7_f05e01670f06.slice/crio-c4b1e5f64254e53231285000f1384d2eb4e389932a78f57212dc19f374ad284e WatchSource:0}: Error finding container c4b1e5f64254e53231285000f1384d2eb4e389932a78f57212dc19f374ad284e: Status 404 returned error can't find the container with id c4b1e5f64254e53231285000f1384d2eb4e389932a78f57212dc19f374ad284e
Nov 02 22:13:43.920278 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920248    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920289    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920345    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920371    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920418    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920437    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920481    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920602    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920628    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfg7s\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920878 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920703    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.920878 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920817    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.921040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920885    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.921040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.920947    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.921266 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.921241    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.923119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.923097    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.923259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.923242    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.977657 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.977619    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:43.978287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:43.978265    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfg7s\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\") pod \"e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") " pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:44.055704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.055658    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:13:44.057418 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:44.057391    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode975a0b5_a29e_4eac_bcfc_0e5530f85cdb.slice/crio-67ed4e6d35963ffe5eda0910ed477c72bc157d277a189bb98833aafb4bb965c2 WatchSource:0}: Error finding container 67ed4e6d35963ffe5eda0910ed477c72bc157d277a189bb98833aafb4bb965c2: Status 404 returned error can't find the container with id 67ed4e6d35963ffe5eda0910ed477c72bc157d277a189bb98833aafb4bb965c2
Nov 02 22:13:44.092279 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.092243    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:44.421426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.421368    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" event={"ID":"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb","Type":"ContainerStarted","Data":"bfcce6809f2fff02cc8f2b28a9390d8fe42457f61b28e46d0796359ca675acf9"}
Nov 02 22:13:44.421904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.421460    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:44.421904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.421484    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" event={"ID":"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb","Type":"ContainerStarted","Data":"67ed4e6d35963ffe5eda0910ed477c72bc157d277a189bb98833aafb4bb965c2"}
Nov 02 22:13:44.423424 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.423396    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" event={"ID":"12236a2c-5fca-46cd-aaa7-f05e01670f06","Type":"ContainerStarted","Data":"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"}
Nov 02 22:13:44.423554 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.423445    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" event={"ID":"12236a2c-5fca-46cd-aaa7-f05e01670f06","Type":"ContainerStarted","Data":"c4b1e5f64254e53231285000f1384d2eb4e389932a78f57212dc19f374ad284e"}
Nov 02 22:13:44.423554 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.423474    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:44.529083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.529048    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:13:44.530314 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:44.530284    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a2beaf_9cdc_417e_9154_ccb65c5a375a.slice/crio-1922531db0aa510acef61b2540114d72b7a484b5f9eca317278094bb5a93ba34 WatchSource:0}: Error finding container 1922531db0aa510acef61b2540114d72b7a484b5f9eca317278094bb5a93ba34: Status 404 returned error can't find the container with id 1922531db0aa510acef61b2540114d72b7a484b5f9eca317278094bb5a93ba34
Nov 02 22:13:44.533157 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.533122    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:44.533260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.533192    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:44.533260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.533228    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:13:44.653688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.653631    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" podStartSLOduration=1.653611369 podStartE2EDuration="1.653611369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:44.652230774 +0000 UTC m=+2323.168726549" watchObservedRunningTime="2026-11-02 22:13:44.653611369 +0000 UTC m=+2323.170107145"
Nov 02 22:13:44.656815 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:44.656747    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" podStartSLOduration=1.6567261210000002 podStartE2EDuration="1.656726121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:44.536631021 +0000 UTC m=+2323.053126796" watchObservedRunningTime="2026-11-02 22:13:44.656726121 +0000 UTC m=+2323.173221896"
Nov 02 22:13:45.432058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:45.432016    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerStarted","Data":"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"}
Nov 02 22:13:45.432476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:45.432085    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerStarted","Data":"1922531db0aa510acef61b2540114d72b7a484b5f9eca317278094bb5a93ba34"}
Nov 02 22:13:45.547877 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:45.547820    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podStartSLOduration=2.547804934 podStartE2EDuration="2.547804934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:45.546599693 +0000 UTC m=+2324.063095491" watchObservedRunningTime="2026-11-02 22:13:45.547804934 +0000 UTC m=+2324.064300708"
Nov 02 22:13:46.093214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:46.093175    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:46.099653 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:46.099625    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:46.438668 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:46.438636    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:46.439869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:46.439841    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:13:51.924796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:51.924755    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:13:51.929635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:51.929611    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.061980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.058169    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9h8\" (UniqueName: \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.061980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.058446    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.061980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.058819    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d1a155-dockercfg-94zfz\""
Nov 02 22:13:52.061980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.058881    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d1a155\""
Nov 02 22:13:52.119589 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.119517    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:13:52.128731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.128703    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:13:52.140757 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.140702    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.160277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.160232    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.160467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.160379    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9h8\" (UniqueName: \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.161592 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.161558    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.162064 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.162044    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:13:52.217137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.217059    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9h8\" (UniqueName: \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\") pod \"payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.244427 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.244397    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:52.262358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.262321    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.262561 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.262535    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxx4q\" (UniqueName: \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.364234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.364181    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxx4q\" (UniqueName: \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.364476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.364457    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.365210 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.365186    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.420937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.420891    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxx4q\" (UniqueName: \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\") pod \"payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.458842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.458798    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:52.621930 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.621893    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:13:52.623216 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:52.623192    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a71af5_1568_4c78_a7cf_70d0eb7defe1.slice/crio-f1c223c40dee3609edecbb7069422b3c2ad6857c1f5f44964ef22081af3e3f00 WatchSource:0}: Error finding container f1c223c40dee3609edecbb7069422b3c2ad6857c1f5f44964ef22081af3e3f00: Status 404 returned error can't find the container with id f1c223c40dee3609edecbb7069422b3c2ad6857c1f5f44964ef22081af3e3f00
Nov 02 22:13:52.838816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:52.838782    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:13:52.840120 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:13:52.840090    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66a6c3f_1624_47d2_876b_96da0b6972a0.slice/crio-bfd4baec301891a5c13ce0c4674d090beceabc110e2833aa444f7d517b03c924 WatchSource:0}: Error finding container bfd4baec301891a5c13ce0c4674d090beceabc110e2833aa444f7d517b03c924: Status 404 returned error can't find the container with id bfd4baec301891a5c13ce0c4674d090beceabc110e2833aa444f7d517b03c924
Nov 02 22:13:53.495377 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.495335    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" event={"ID":"e8a71af5-1568-4c78-a7cf-70d0eb7defe1","Type":"ContainerStarted","Data":"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"}
Nov 02 22:13:53.495839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.495408    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" event={"ID":"e8a71af5-1568-4c78-a7cf-70d0eb7defe1","Type":"ContainerStarted","Data":"f1c223c40dee3609edecbb7069422b3c2ad6857c1f5f44964ef22081af3e3f00"}
Nov 02 22:13:53.495839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.495438    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:13:53.497324 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.497298    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" event={"ID":"c66a6c3f-1624-47d2-876b-96da0b6972a0","Type":"ContainerStarted","Data":"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"}
Nov 02 22:13:53.497483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.497341    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" event={"ID":"c66a6c3f-1624-47d2-876b-96da0b6972a0","Type":"ContainerStarted","Data":"bfd4baec301891a5c13ce0c4674d090beceabc110e2833aa444f7d517b03c924"}
Nov 02 22:13:53.497483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.497430    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:13:53.609791 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.609722    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" podStartSLOduration=2.609703929 podStartE2EDuration="2.609703929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:53.606507838 +0000 UTC m=+2332.123003612" watchObservedRunningTime="2026-11-02 22:13:53.609703929 +0000 UTC m=+2332.126199705"
Nov 02 22:13:53.728246 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:53.728183    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" podStartSLOduration=1.7281612100000001 podStartE2EDuration="1.72816121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:53.72293915 +0000 UTC m=+2332.239434936" watchObservedRunningTime="2026-11-02 22:13:53.72816121 +0000 UTC m=+2332.244656985"
Nov 02 22:13:55.295629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:55.295588    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:13:55.435385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:55.435354    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:13:55.435554 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:13:55.435440    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:13:56.312268 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:13:56.312222    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:14:02.954935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:02.954899    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:14:02.955477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:02.955176    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy" containerID="cri-o://0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117" gracePeriod=30
Nov 02 22:14:04.508391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:04.508348    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:14:04.508917 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:04.508445    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:14:08.232243 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.232161    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:14:08.327073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327030    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327088    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327205    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327397 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327315    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327395    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327398    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:08.327454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327444    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:08.327610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327510    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327547    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327616    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327665    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\") pod \"38a90e63-7fc0-4870-8b0e-7005d84b5414\" (UID: \"38a90e63-7fc0-4870-8b0e-7005d84b5414\") "
Nov 02 22:14:08.327826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327784    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:08.327884 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.327816    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:14:08.328242 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328196    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:08.328366 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328326    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.328366 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328347    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.328366 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328363    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38a90e63-7fc0-4870-8b0e-7005d84b5414-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.328526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328378    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.328526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.328393    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.330308 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.330280    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:08.330433 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.330389    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "kube-api-access-jntst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:08.330549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.330528    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:08.332111 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.332085    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo" pod "38a90e63-7fc0-4870-8b0e-7005d84b5414" (UID: "38a90e63-7fc0-4870-8b0e-7005d84b5414"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:14:08.431233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.431192    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jntst\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-kube-api-access-jntst\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.431233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.431236    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.431479 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.431252    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.431479 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.431266    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38a90e63-7fc0-4870-8b0e-7005d84b5414-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:08.644352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.644250    5580 generic.go:356] "Generic (PLEG): container finished" podID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerID="0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117" exitCode=0
Nov 02 22:14:08.644352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.644300    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"
Nov 02 22:14:08.644352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.644330    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerDied","Data":"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"}
Nov 02 22:14:08.644649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.644383    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6" event={"ID":"38a90e63-7fc0-4870-8b0e-7005d84b5414","Type":"ContainerDied","Data":"036b12871f27fc2603e013149f095bb168f1689c68d8919c262c091b8dcd8232"}
Nov 02 22:14:08.644649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.644406    5580 scope.go:122] "RemoveContainer" containerID="0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"
Nov 02 22:14:08.656009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.655991    5580 scope.go:122] "RemoveContainer" containerID="7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"
Nov 02 22:14:08.665678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.665652    5580 scope.go:122] "RemoveContainer" containerID="0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"
Nov 02 22:14:08.666216 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:08.666185    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117\": container with ID starting with 0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117 not found: ID does not exist" containerID="0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"
Nov 02 22:14:08.666328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.666226    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117"} err="failed to get container status \"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117\": rpc error: code = NotFound desc = could not find container \"0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117\": container with ID starting with 0a40bc314fd1648ee342a37fa5752b6cbd59594d784ffd44ddf4b465be114117 not found: ID does not exist"
Nov 02 22:14:08.666328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.666247    5580 scope.go:122] "RemoveContainer" containerID="7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"
Nov 02 22:14:08.666595 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:08.666572    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f\": container with ID starting with 7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f not found: ID does not exist" containerID="7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"
Nov 02 22:14:08.666664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.666603    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f"} err="failed to get container status \"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f\": rpc error: code = NotFound desc = could not find container \"7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f\": container with ID starting with 7c1c042d6cca97c1e6f666436efc9abfc0b254c0793066787d8c5da63eef251f not found: ID does not exist"
Nov 02 22:14:08.809134 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.809095    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:14:08.811925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:08.811897    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-027bdf-openshift-default-5cbf665885-6gjq6"]
Nov 02 22:14:09.529995 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:09.528514    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 22:14:09.595087 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:09.595050    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" path="/var/lib/kubelet/pods/38a90e63-7fc0-4870-8b0e-7005d84b5414/volumes"
Nov 02 22:14:14.799407 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:14.799320    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 22:14:17.329535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.329497    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 22:14:17.330092 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.329722    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerName="maas-api" containerID="cri-o://f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9" gracePeriod=30
Nov 02 22:14:17.653667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.653636    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 22:14:17.760428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.760392    5580 generic.go:356] "Generic (PLEG): container finished" podID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerID="f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9" exitCode=0
Nov 02 22:14:17.760642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.760483    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" event={"ID":"ba18e99d-dd3b-4176-a6ed-741b645187dd","Type":"ContainerDied","Data":"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"}
Nov 02 22:14:17.760642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.760548    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" event={"ID":"ba18e99d-dd3b-4176-a6ed-741b645187dd","Type":"ContainerDied","Data":"7c239d0542b7c130b64ba4603f09d445b0c9e518e883961c732e77470b9398bc"}
Nov 02 22:14:17.760642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.760572    5580 scope.go:122] "RemoveContainer" containerID="f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"
Nov 02 22:14:17.760818 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.760708    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"
Nov 02 22:14:17.775324 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.775291    5580 scope.go:122] "RemoveContainer" containerID="f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"
Nov 02 22:14:17.775773 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:17.775739    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9\": container with ID starting with f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9 not found: ID does not exist" containerID="f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"
Nov 02 22:14:17.775906 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.775786    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9"} err="failed to get container status \"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9\": rpc error: code = NotFound desc = could not find container \"f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9\": container with ID starting with f7d50f4491ac5c2132b77a4d23781ea0caf6a3d59b1167eceffb0931989ff8f9 not found: ID does not exist"
Nov 02 22:14:17.785402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.785370    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\") pod \"ba18e99d-dd3b-4176-a6ed-741b645187dd\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") "
Nov 02 22:14:17.785607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.785513    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\") pod \"ba18e99d-dd3b-4176-a6ed-741b645187dd\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") "
Nov 02 22:14:17.785607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.785560    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") pod \"ba18e99d-dd3b-4176-a6ed-741b645187dd\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") "
Nov 02 22:14:17.785607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.785602    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\" (UniqueName: \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\") pod \"ba18e99d-dd3b-4176-a6ed-741b645187dd\" (UID: \"ba18e99d-dd3b-4176-a6ed-741b645187dd\") "
Nov 02 22:14:17.786194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.786159    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca" pod "ba18e99d-dd3b-4176-a6ed-741b645187dd" (UID: "ba18e99d-dd3b-4176-a6ed-741b645187dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:14:17.788438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.788412    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls" pod "ba18e99d-dd3b-4176-a6ed-741b645187dd" (UID: "ba18e99d-dd3b-4176-a6ed-741b645187dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:17.788581 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.788555    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd" pod "ba18e99d-dd3b-4176-a6ed-741b645187dd" (UID: "ba18e99d-dd3b-4176-a6ed-741b645187dd"). InnerVolumeSpecName "kube-api-access-n8cqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:17.788581 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.788437    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs" pod "ba18e99d-dd3b-4176-a6ed-741b645187dd" (UID: "ba18e99d-dd3b-4176-a6ed-741b645187dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:17.888105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.888005    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8cqd\" (UniqueName: \"kubernetes.io/projected/ba18e99d-dd3b-4176-a6ed-741b645187dd-kube-api-access-n8cqd\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:17.888105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.888041    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba18e99d-dd3b-4176-a6ed-741b645187dd-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:17.888105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.888053    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:17.888105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:17.888062    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba18e99d-dd3b-4176-a6ed-741b645187dd-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:18.193358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:18.193321    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 22:14:18.196121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:18.196089    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh"]
Nov 02 22:14:19.597813 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:19.597772    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" path="/var/lib/kubelet/pods/ba18e99d-dd3b-4176-a6ed-741b645187dd/volumes"
Nov 02 22:14:20.534076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:20.534027    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f48c4d-bd7686d66-9rzkh" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.43:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:14:24.571633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:24.571580    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main" containerID="cri-o://323b9492bc60e0a297e9544fb1b9891fc68ebd7c07be634db0dc9c86afe2a181" gracePeriod=45
Nov 02 22:14:24.821328 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:24.821290    5580 generic.go:356] "Generic (PLEG): container finished" podID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerID="323b9492bc60e0a297e9544fb1b9891fc68ebd7c07be634db0dc9c86afe2a181" exitCode=0
Nov 02 22:14:24.821799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:24.821715    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" event={"ID":"fc56242a-50bb-4a30-9e30-6e4b9b99b04a","Type":"ContainerDied","Data":"323b9492bc60e0a297e9544fb1b9891fc68ebd7c07be634db0dc9c86afe2a181"}
Nov 02 22:14:24.935779 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:24.935746    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 22:14:25.054625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054446    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.054625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054594    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.054932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054762    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.054932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054879    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.054932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054919    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\" (UniqueName: \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.055109 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.054951    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\") pod \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\" (UID: \"fc56242a-50bb-4a30-9e30-6e4b9b99b04a\") "
Nov 02 22:14:25.056037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.056010    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:25.056381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.056361    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:25.059982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.059931    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:25.060171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.059999    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:25.062483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.062448    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "kube-api-access-lsfct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:25.062630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.062498    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs" pod "fc56242a-50bb-4a30-9e30-6e4b9b99b04a" (UID: "fc56242a-50bb-4a30-9e30-6e4b9b99b04a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:25.156836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156794    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.156836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156829    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.156836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156842    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.157139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156852    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsfct\" (UniqueName: \"kubernetes.io/projected/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-kube-api-access-lsfct\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.157139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156864    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.157139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.156874    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc56242a-50bb-4a30-9e30-6e4b9b99b04a-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:25.831695 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.831663    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"
Nov 02 22:14:25.831695 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.831691    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt" event={"ID":"fc56242a-50bb-4a30-9e30-6e4b9b99b04a","Type":"ContainerDied","Data":"5ba48a1405980b4903cfc2b77c7cfa2b55f969d3a42dd299bd32eeeffa5f7642"}
Nov 02 22:14:25.832262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.831746    5580 scope.go:122] "RemoveContainer" containerID="323b9492bc60e0a297e9544fb1b9891fc68ebd7c07be634db0dc9c86afe2a181"
Nov 02 22:14:25.994505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:25.994470    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 22:14:26.043876 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:26.043835    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-fb8112/distinct-2-w6-fb8112-kserve-67897574d8-pb8qt"]
Nov 02 22:14:26.322641 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:26.322549    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:14:27.448295 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:27.448255    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 22:14:27.497595 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:27.497558    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 22:14:27.594639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:27.594602    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" path="/var/lib/kubelet/pods/fc56242a-50bb-4a30-9e30-6e4b9b99b04a/volumes"
Nov 02 22:14:29.821802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:29.821750    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main" containerID="cri-o://41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3" gracePeriod=45
Nov 02 22:14:30.104012 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.103981    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 22:14:30.229096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229065    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229139    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229178    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229226    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\" (UniqueName: \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229331    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:30.229457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229376    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229431    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:30.229605 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229491    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:30.229605 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229515    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\" (UniqueName: \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\") pod \"454a151c-25fd-485e-8503-ec363d932a36\" (UID: \"454a151c-25fd-485e-8503-ec363d932a36\") "
Nov 02 22:14:30.229991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229952    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.229991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.229993    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.230197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.230006    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.231756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.231734    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:30.231859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.231762    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "kube-api-access-4xmhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:30.231859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.231773    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs" pod "454a151c-25fd-485e-8503-ec363d932a36" (UID: "454a151c-25fd-485e-8503-ec363d932a36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:30.331883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.331775    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/454a151c-25fd-485e-8503-ec363d932a36-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.331883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.331815    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/454a151c-25fd-485e-8503-ec363d932a36-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.331883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.331832    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xmhm\" (UniqueName: \"kubernetes.io/projected/454a151c-25fd-485e-8503-ec363d932a36-kube-api-access-4xmhm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:30.869974 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.869925    5580 generic.go:356] "Generic (PLEG): container finished" podID="454a151c-25fd-485e-8503-ec363d932a36" containerID="41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3" exitCode=0
Nov 02 22:14:30.870467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.869980    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" event={"ID":"454a151c-25fd-485e-8503-ec363d932a36","Type":"ContainerDied","Data":"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"}
Nov 02 22:14:30.870467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.870006    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"
Nov 02 22:14:30.870467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.870037    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6" event={"ID":"454a151c-25fd-485e-8503-ec363d932a36","Type":"ContainerDied","Data":"f089141d8d63d0a876bbf4464abf6036868cc51b4b6e34f5fc487ed180b7cc7d"}
Nov 02 22:14:30.870467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.870060    5580 scope.go:122] "RemoveContainer" containerID="41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"
Nov 02 22:14:30.883026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.883008    5580 scope.go:122] "RemoveContainer" containerID="41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"
Nov 02 22:14:30.883356 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:30.883334    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3\": container with ID starting with 41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3 not found: ID does not exist" containerID="41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"
Nov 02 22:14:30.883437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:30.883365    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3"} err="failed to get container status \"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3\": rpc error: code = NotFound desc = could not find container \"41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3\": container with ID starting with 41bf684fb8e0d5dbd137eeb2cf21d4d596dbefc0f099bd673e0dee7e88e85ad3 not found: ID does not exist"
Nov 02 22:14:31.031481 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:31.031443    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 22:14:31.081503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:31.081464    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-64438f/premium-w3-64438f-kserve-75dd89888-hqsx6"]
Nov 02 22:14:31.599514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:31.599474    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="454a151c-25fd-485e-8503-ec363d932a36" path="/var/lib/kubelet/pods/454a151c-25fd-485e-8503-ec363d932a36/volumes"
Nov 02 22:14:37.327139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:37.327090    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:14:37.327941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:37.327902    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:14:37.387688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:37.387636    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:14:37.388046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:37.388003    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:14:42.472883 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.472835    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" containerID="cri-o://cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95" gracePeriod=45
Nov 02 22:14:42.519498 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.519452    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" containerID="cri-o://20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411" gracePeriod=45
Nov 02 22:14:42.678191 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.678142    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": dial tcp 10.134.0.51:8000: connect: connection refused"
Nov 02 22:14:42.800101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.800069    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 22:14:42.825842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.825810    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 22:14:42.936057 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936018    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936172    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936219    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.936423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936282    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.936423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936338    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936372    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936377    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936469    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\" (UniqueName: \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936488    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936542    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936573    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\" (UniqueName: \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936563    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.936643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936608    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.936986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936636    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.936986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936654    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\" (UniqueName: \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\") pod \"d5604e3f-1005-4c74-be6e-6ec235828e9b\" (UID: \"d5604e3f-1005-4c74-be6e-6ec235828e9b\") "
Nov 02 22:14:42.936986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936715    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.936986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.936746    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\") pod \"823c7335-280d-4641-8cc1-f6deec541e0f\" (UID: \"823c7335-280d-4641-8cc1-f6deec541e0f\") "
Nov 02 22:14:42.937537 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937508    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:42.937656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937542    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:42.937656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937557    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:42.937656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937569    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:42.937656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937582    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:42.937873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.937825    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.940006 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.939977    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:42.940140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.940122    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.940510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.940473    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k" pod "823c7335-280d-4641-8cc1-f6deec541e0f" (UID: "823c7335-280d-4641-8cc1-f6deec541e0f"). InnerVolumeSpecName "kube-api-access-jkz6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:42.940652 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.940620    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "kube-api-access-zfdq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:42.940727 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.940644    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:14:42.942377 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:42.942344    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs" pod "d5604e3f-1005-4c74-be6e-6ec235828e9b" (UID: "d5604e3f-1005-4c74-be6e-6ec235828e9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038460    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5604e3f-1005-4c74-be6e-6ec235828e9b-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038495    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkz6k\" (UniqueName: \"kubernetes.io/projected/823c7335-280d-4641-8cc1-f6deec541e0f-kube-api-access-jkz6k\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038511    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5604e3f-1005-4c74-be6e-6ec235828e9b-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038525    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823c7335-280d-4641-8cc1-f6deec541e0f-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038536    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038548 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038548    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfdq5\" (UniqueName: \"kubernetes.io/projected/d5604e3f-1005-4c74-be6e-6ec235828e9b-kube-api-access-zfdq5\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.038914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.038560    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823c7335-280d-4641-8cc1-f6deec541e0f-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:14:43.084066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.084027    5580 generic.go:356] "Generic (PLEG): container finished" podID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerID="cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95" exitCode=0
Nov 02 22:14:43.084273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.084095    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"
Nov 02 22:14:43.084273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.084132    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" event={"ID":"d5604e3f-1005-4c74-be6e-6ec235828e9b","Type":"ContainerDied","Data":"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"}
Nov 02 22:14:43.084273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.084213    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" event={"ID":"d5604e3f-1005-4c74-be6e-6ec235828e9b","Type":"ContainerDied","Data":"30ff02e20d7473e3775ae7912e00128893e14444f1ddc8ec0124ec05a0c2dc60"}
Nov 02 22:14:43.084273 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.084239    5580 scope.go:122] "RemoveContainer" containerID="cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"
Nov 02 22:14:43.086827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.086786    5580 generic.go:356] "Generic (PLEG): container finished" podID="823c7335-280d-4641-8cc1-f6deec541e0f" containerID="20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411" exitCode=0
Nov 02 22:14:43.087038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.086836    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" event={"ID":"823c7335-280d-4641-8cc1-f6deec541e0f","Type":"ContainerDied","Data":"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"}
Nov 02 22:14:43.087038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.086869    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d" event={"ID":"823c7335-280d-4641-8cc1-f6deec541e0f","Type":"ContainerDied","Data":"3cf618aeb061ed503d8e3af0ef782754749cc544833f11dda6fec555ead3fd57"}
Nov 02 22:14:43.087038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.086930    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"
Nov 02 22:14:43.097714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.097692    5580 scope.go:122] "RemoveContainer" containerID="cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"
Nov 02 22:14:43.098230 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:43.098126    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95\": container with ID starting with cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95 not found: ID does not exist" containerID="cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"
Nov 02 22:14:43.098230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.098168    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95"} err="failed to get container status \"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95\": rpc error: code = NotFound desc = could not find container \"cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95\": container with ID starting with cc190375b24e7b0049771c95711f7b9c665ab67d6028bbd2d4f7eb24d9b7cb95 not found: ID does not exist"
Nov 02 22:14:43.098230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.098197    5580 scope.go:122] "RemoveContainer" containerID="20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"
Nov 02 22:14:43.109590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.109565    5580 scope.go:122] "RemoveContainer" containerID="20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"
Nov 02 22:14:43.109869 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:43.109847    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411\": container with ID starting with 20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411 not found: ID does not exist" containerID="20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"
Nov 02 22:14:43.109942 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.109882    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411"} err="failed to get container status \"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411\": rpc error: code = NotFound desc = could not find container \"20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411\": container with ID starting with 20c220b631392f470d6c41f7244b113067c432c7820bf4b7727d92171bdbb411 not found: ID does not exist"
Nov 02 22:14:43.247969 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.247919    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 22:14:43.251755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.251724    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb"]
Nov 02 22:14:43.358798 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.358707    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 22:14:43.362159 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.362132    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ffe386/distinct-w1-ffe386-kserve-57c6bdb578-vbk9d"]
Nov 02 22:14:43.594848 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.594814    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" path="/var/lib/kubelet/pods/823c7335-280d-4641-8cc1-f6deec541e0f/volumes"
Nov 02 22:14:43.595336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.595322    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" path="/var/lib/kubelet/pods/d5604e3f-1005-4c74-be6e-6ec235828e9b/volumes"
Nov 02 22:14:43.746802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:14:43.746756    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ffe386/distinct-2-w1-ffe386-kserve-747d659b57-ll4mb" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:14:56.313077 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:14:56.313038    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:15:00.188838 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.188789    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:00.189774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189750    5580 cpu_manager.go:414] "containerMap: removing container" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.189827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189778    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.189827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189792    5580 cpu_manager.go:414] "containerMap: removing container" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main"
Nov 02 22:15:00.189827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189799    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main"
Nov 02 22:15:00.189827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189820    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189828    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189872    5580 cpu_manager.go:414] "containerMap: removing container" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189880    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189891    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerName="maas-api"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189899    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerName="maas-api"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189925    5580 cpu_manager.go:414] "containerMap: removing container" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189931    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189944    5580 cpu_manager.go:414] "containerMap: removing container" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.190003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.189951    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190134    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc56242a-50bb-4a30-9e30-6e4b9b99b04a" containerName="main"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190148    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba18e99d-dd3b-4176-a6ed-741b645187dd" containerName="maas-api"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190163    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="454a151c-25fd-485e-8503-ec363d932a36" containerName="main"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190174    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190188    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="38a90e63-7fc0-4870-8b0e-7005d84b5414" containerName="istio-proxy"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190211    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="823c7335-280d-4641-8cc1-f6deec541e0f" containerName="main"
Nov 02 22:15:00.190341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.190222    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5604e3f-1005-4c74-be6e-6ec235828e9b" containerName="main"
Nov 02 22:15:00.201177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.201145    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:00.201381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.201267    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.206271 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.206242    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.209555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.209525    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:00.214324 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.214302    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.253659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.253141    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c4p9p\""
Nov 02 22:15:00.315578 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.315540    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:00.321625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.321588    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svc7h\" (UniqueName: \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.321826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.321648    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.321826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.321738    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.321949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.321902    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.322072 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.322050    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7bj\" (UniqueName: \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.322194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.322163    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4snl7\" (UniqueName: \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.363502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.363461    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:00.367024 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.366986    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:00.423227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423185    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7bj\" (UniqueName: \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.423441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423251    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4snl7\" (UniqueName: \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.423441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423336    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svc7h\" (UniqueName: \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.423441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423360    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.423441 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423394    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.423689 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.423475    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.424180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.424156    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.424180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.424167    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.424334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.424167    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.479898 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.479813    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svc7h\" (UniqueName: \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.480085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.479918    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7bj\" (UniqueName: \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\") pod \"maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.480085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.480000    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4snl7\" (UniqueName: \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\") pod \"maas-api-key-cleanup-29894295-9nplw\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.518782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.518742    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:00.526549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.526513    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:00.533825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.533795    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:00.909398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.909312    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:00.913596 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:15:00.913566    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885393f9_b131_4c1b_a4d1_0c9d110f6e7e.slice/crio-6b04fb548831d6d281ee520db1a6820e3267bca5c71cd20b3d20ad4796dbfcdd WatchSource:0}: Error finding container 6b04fb548831d6d281ee520db1a6820e3267bca5c71cd20b3d20ad4796dbfcdd: Status 404 returned error can't find the container with id 6b04fb548831d6d281ee520db1a6820e3267bca5c71cd20b3d20ad4796dbfcdd
Nov 02 22:15:00.916215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.916197    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:15:00.961214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.959730    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:00.963023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:00.962868    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:00.963503 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:15:00.963404    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c5327e_99aa_4432_b8d5_741b438e0929.slice/crio-2c55d224425da9ae4a874861c9daa355979bd6c0086e00e4fc4c3f7d9776c229 WatchSource:0}: Error finding container 2c55d224425da9ae4a874861c9daa355979bd6c0086e00e4fc4c3f7d9776c229: Status 404 returned error can't find the container with id 2c55d224425da9ae4a874861c9daa355979bd6c0086e00e4fc4c3f7d9776c229
Nov 02 22:15:00.966413 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:15:00.966375    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05352888_6b6c_4a3c_95ec_a3d83f40cd0d.slice/crio-3d8dfe1a12bc624d41567a237eb86cbbefb1a7c8af1b11b52b705fa4c125145e WatchSource:0}: Error finding container 3d8dfe1a12bc624d41567a237eb86cbbefb1a7c8af1b11b52b705fa4c125145e: Status 404 returned error can't find the container with id 3d8dfe1a12bc624d41567a237eb86cbbefb1a7c8af1b11b52b705fa4c125145e
Nov 02 22:15:01.216036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:01.215993    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerStarted","Data":"6b04fb548831d6d281ee520db1a6820e3267bca5c71cd20b3d20ad4796dbfcdd"}
Nov 02 22:15:01.217719 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:01.217683    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerStarted","Data":"2c55d224425da9ae4a874861c9daa355979bd6c0086e00e4fc4c3f7d9776c229"}
Nov 02 22:15:01.219289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:01.219259    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerStarted","Data":"3d8dfe1a12bc624d41567a237eb86cbbefb1a7c8af1b11b52b705fa4c125145e"}
Nov 02 22:15:02.181998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:02.181952    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:15:02.189936 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:02.189910    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:15:02.193194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:02.193167    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:15:02.200842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:02.200818    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:15:03.714000 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:03.713949    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:15:07.272121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:07.272074    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerStarted","Data":"21c14eb1bf047bd179d2e7a27a4824f12bd14d0cf58a35d5adc774ce55c0c96e"}
Nov 02 22:15:07.428396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:07.428330    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" podStartSLOduration=1.709782127 podStartE2EDuration="7.428313607s" totalImagesPullingTime="5.71853148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:15:07.421437305 +0000 UTC m=+2405.937933080" watchObservedRunningTime="2026-11-02 22:15:07.428313607 +0000 UTC m=+2405.944809381"
Nov 02 22:15:08.333697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:08.333660    5580 generic.go:356] "Generic (PLEG): container finished" podID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerID="21c14eb1bf047bd179d2e7a27a4824f12bd14d0cf58a35d5adc774ce55c0c96e" exitCode=60
Nov 02 22:15:08.334212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:08.333746    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerDied","Data":"21c14eb1bf047bd179d2e7a27a4824f12bd14d0cf58a35d5adc774ce55c0c96e"}
Nov 02 22:15:08.334212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:08.334100    5580 scope.go:122] "RemoveContainer" containerID="21c14eb1bf047bd179d2e7a27a4824f12bd14d0cf58a35d5adc774ce55c0c96e"
Nov 02 22:15:09.343926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.343887    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerStarted","Data":"ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"}
Nov 02 22:15:09.345615 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.345590    5580 generic.go:356] "Generic (PLEG): container finished" podID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerID="4a6fbd3ae9dac6897297d78b5b6da38dcce08f11c3d92fd98c10cb3770a98fbc" exitCode=60
Nov 02 22:15:09.345723 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.345662    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerDied","Data":"4a6fbd3ae9dac6897297d78b5b6da38dcce08f11c3d92fd98c10cb3770a98fbc"}
Nov 02 22:15:09.346274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.346257    5580 scope.go:122] "RemoveContainer" containerID="4a6fbd3ae9dac6897297d78b5b6da38dcce08f11c3d92fd98c10cb3770a98fbc"
Nov 02 22:15:09.347264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.347244    5580 generic.go:356] "Generic (PLEG): container finished" podID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerID="55d32a533442e454f7a6ddd038c2c60abd0314d4143d03e080f5dd978e7ce873" exitCode=60
Nov 02 22:15:09.347360 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.347343    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerDied","Data":"55d32a533442e454f7a6ddd038c2c60abd0314d4143d03e080f5dd978e7ce873"}
Nov 02 22:15:09.347622 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:09.347608    5580 scope.go:122] "RemoveContainer" containerID="55d32a533442e454f7a6ddd038c2c60abd0314d4143d03e080f5dd978e7ce873"
Nov 02 22:15:10.356794 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.356756    5580 generic.go:356] "Generic (PLEG): container finished" podID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerID="ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8" exitCode=60
Nov 02 22:15:10.357272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.356833    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerDied","Data":"ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"}
Nov 02 22:15:10.357272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.356891    5580 scope.go:122] "RemoveContainer" containerID="21c14eb1bf047bd179d2e7a27a4824f12bd14d0cf58a35d5adc774ce55c0c96e"
Nov 02 22:15:10.357272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.357232    5580 scope.go:122] "RemoveContainer" containerID="ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"
Nov 02 22:15:10.357805 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:10.357602    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894295-9nplw_odh-ai-gateway-infra(885393f9-b131-4c1b-a4d1-0c9d110f6e7e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e"
Nov 02 22:15:10.359249 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.359217    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerStarted","Data":"416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"}
Nov 02 22:15:10.361322 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.361295    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerStarted","Data":"5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"}
Nov 02 22:15:10.583332 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.582940    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" podStartSLOduration=3.691618374 podStartE2EDuration="10.582921009s" totalImagesPullingTime="6.891302635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:15:10.582199177 +0000 UTC m=+2409.098694950" watchObservedRunningTime="2026-11-02 22:15:10.582921009 +0000 UTC m=+2409.099416784"
Nov 02 22:15:10.696751 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:10.696684    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" podStartSLOduration=3.926714751 podStartE2EDuration="10.69666046s" totalImagesPullingTime="6.769945709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:15:10.693937235 +0000 UTC m=+2409.210433008" watchObservedRunningTime="2026-11-02 22:15:10.69666046 +0000 UTC m=+2409.213156235"
Nov 02 22:15:11.370860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.370827    5580 scope.go:122] "RemoveContainer" containerID="ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"
Nov 02 22:15:11.371390 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:11.371236    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894295-9nplw_odh-ai-gateway-infra(885393f9-b131-4c1b-a4d1-0c9d110f6e7e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e"
Nov 02 22:15:11.373290 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.373260    5580 generic.go:356] "Generic (PLEG): container finished" podID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerID="416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f" exitCode=60
Nov 02 22:15:11.373462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.373339    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerDied","Data":"416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"}
Nov 02 22:15:11.373462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.373411    5580 scope.go:122] "RemoveContainer" containerID="4a6fbd3ae9dac6897297d78b5b6da38dcce08f11c3d92fd98c10cb3770a98fbc"
Nov 02 22:15:11.373836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.373815    5580 scope.go:122] "RemoveContainer" containerID="416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"
Nov 02 22:15:11.374294 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:11.374266    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt_odh-ai-gateway-infra(b5c5327e-99aa-4432-b8d5-741b438e0929)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929"
Nov 02 22:15:11.376430 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.376400    5580 generic.go:356] "Generic (PLEG): container finished" podID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerID="5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3" exitCode=60
Nov 02 22:15:11.376553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.376466    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerDied","Data":"5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"}
Nov 02 22:15:11.376804 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.376789    5580 scope.go:122] "RemoveContainer" containerID="5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"
Nov 02 22:15:11.377201 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:11.377178    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf_odh-ai-gateway-infra(05352888-6b6c-4a3c-95ec-a3d83f40cd0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d"
Nov 02 22:15:11.389660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:11.389630    5580 scope.go:122] "RemoveContainer" containerID="55d32a533442e454f7a6ddd038c2c60abd0314d4143d03e080f5dd978e7ce873"
Nov 02 22:15:12.385429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:12.385393    5580 scope.go:122] "RemoveContainer" containerID="416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"
Nov 02 22:15:12.385868 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:12.385787    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt_odh-ai-gateway-infra(b5c5327e-99aa-4432-b8d5-741b438e0929)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929"
Nov 02 22:15:12.388562 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:12.388539    5580 scope.go:122] "RemoveContainer" containerID="5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"
Nov 02 22:15:12.388864 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:12.388845    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf_odh-ai-gateway-infra(05352888-6b6c-4a3c-95ec-a3d83f40cd0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d"
Nov 02 22:15:14.405020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.404978    5580 generic.go:356] "Generic (PLEG): container finished" podID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerID="50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936" exitCode=0
Nov 02 22:15:14.405538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.405069    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerDied","Data":"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"}
Nov 02 22:15:14.405610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.405575    5580 scope.go:122] "RemoveContainer" containerID="50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"
Nov 02 22:15:14.407236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407213    5580 generic.go:356] "Generic (PLEG): container finished" podID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerID="d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d" exitCode=0
Nov 02 22:15:14.407302 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407271    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerDied","Data":"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"}
Nov 02 22:15:14.407480 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407453    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:14.407556 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407543    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:14.407594 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407580    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:14.407627 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.407614    5580 scope.go:122] "RemoveContainer" containerID="d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"
Nov 02 22:15:14.408881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.408853    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:14.409009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.408915    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:14.409009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:14.408949    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:15:15.435676 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:15.435646    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerStarted","Data":"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"}
Nov 02 22:15:15.437918 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:15.437892    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerStarted","Data":"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"}
Nov 02 22:15:16.098247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.098144    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:15:16.104211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.104150    5580 patch_prober.go:29] interesting pod/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Nov 02 22:15:16.104543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.104495    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Nov 02 22:15:16.407657 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.407562    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:15:16.408586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.408541    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 02 22:15:16.408816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.408790    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 02 22:15:16.439429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:16.439227    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:15:17.096373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:17.096330    5580 patch_prober.go:29] interesting pod/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Nov 02 22:15:17.096688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:17.096652    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Nov 02 22:15:17.403575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:17.403473    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 02 22:15:17.403924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:17.403884    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 02 22:15:18.098469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:18.098427    5580 patch_prober.go:29] interesting pod/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Nov 02 22:15:18.099152 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:18.099116    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Nov 02 22:15:18.403577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:18.403481    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 02 22:15:18.404990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:18.404930    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 02 22:15:19.097997 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:19.094423    5580 patch_prober.go:29] interesting pod/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Nov 02 22:15:19.097997 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:19.094491    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Nov 02 22:15:19.412229 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:19.412127    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 02 22:15:19.412825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:19.412788    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 02 22:15:20.097236 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:20.097195    5580 patch_prober.go:29] interesting pod/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Nov 02 22:15:20.097553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:20.097519    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Nov 02 22:15:20.404051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:20.403405    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 02 22:15:20.404051 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:20.403473    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 02 22:15:21.097842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.097805    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:15:21.099581 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.099552    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:15:21.408682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.408590    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:15:21.409050 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.409019    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:15:21.410351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.410325    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:15:21.682474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.682434    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 22:15:21.682758 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.682708    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" podUID="91a8e487-5b23-4a12-8cda-0c8e8988779d" containerName="maas-api" containerID="cri-o://3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396" gracePeriod=30
Nov 02 22:15:21.983792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:21.983758    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 22:15:22.166281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.166239    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\") pod \"91a8e487-5b23-4a12-8cda-0c8e8988779d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") "
Nov 02 22:15:22.166781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.166512    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\") pod \"91a8e487-5b23-4a12-8cda-0c8e8988779d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") "
Nov 02 22:15:22.166781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.166600    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") pod \"91a8e487-5b23-4a12-8cda-0c8e8988779d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") "
Nov 02 22:15:22.166781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.166631    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\" (UniqueName: \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\") pod \"91a8e487-5b23-4a12-8cda-0c8e8988779d\" (UID: \"91a8e487-5b23-4a12-8cda-0c8e8988779d\") "
Nov 02 22:15:22.167005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.166939    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca" pod "91a8e487-5b23-4a12-8cda-0c8e8988779d" (UID: "91a8e487-5b23-4a12-8cda-0c8e8988779d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:22.167190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.167175    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a8e487-5b23-4a12-8cda-0c8e8988779d-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:22.168483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.168458    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs" pod "91a8e487-5b23-4a12-8cda-0c8e8988779d" (UID: "91a8e487-5b23-4a12-8cda-0c8e8988779d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:15:22.168659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.168645    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls" pod "91a8e487-5b23-4a12-8cda-0c8e8988779d" (UID: "91a8e487-5b23-4a12-8cda-0c8e8988779d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:15:22.168873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.168848    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r" pod "91a8e487-5b23-4a12-8cda-0c8e8988779d" (UID: "91a8e487-5b23-4a12-8cda-0c8e8988779d"). InnerVolumeSpecName "kube-api-access-hn74r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:22.268352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.268251    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:22.268352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.268293    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn74r\" (UniqueName: \"kubernetes.io/projected/91a8e487-5b23-4a12-8cda-0c8e8988779d-kube-api-access-hn74r\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:22.268352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.268327    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a8e487-5b23-4a12-8cda-0c8e8988779d-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:22.503707 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.503667    5580 generic.go:356] "Generic (PLEG): container finished" podID="91a8e487-5b23-4a12-8cda-0c8e8988779d" containerID="3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396" exitCode=0
Nov 02 22:15:22.503915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.503720    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"
Nov 02 22:15:22.503915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.503753    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" event={"ID":"91a8e487-5b23-4a12-8cda-0c8e8988779d","Type":"ContainerDied","Data":"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"}
Nov 02 22:15:22.503915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.503811    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d" event={"ID":"91a8e487-5b23-4a12-8cda-0c8e8988779d","Type":"ContainerDied","Data":"b732477b29309c34576b5a27b45c5bcaa2b6675ec8c1d84840896b4953c1cc3f"}
Nov 02 22:15:22.503915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.503833    5580 scope.go:122] "RemoveContainer" containerID="3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"
Nov 02 22:15:22.514706 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.514685    5580 scope.go:122] "RemoveContainer" containerID="3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"
Nov 02 22:15:22.515015 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:22.514996    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396\": container with ID starting with 3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396 not found: ID does not exist" containerID="3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"
Nov 02 22:15:22.515076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.515026    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396"} err="failed to get container status \"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396\": rpc error: code = NotFound desc = could not find container \"3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396\": container with ID starting with 3059a83e5e52f3e19acc8a4ffb319e065da78226adce569bb0dc3c438ae3b396 not found: ID does not exist"
Nov 02 22:15:22.694998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.691022    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 22:15:22.708779 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:22.708725    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-fb8112-5b887459cf-gqw6d"]
Nov 02 22:15:23.586944 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:23.586909    5580 scope.go:122] "RemoveContainer" containerID="ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"
Nov 02 22:15:23.587410 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:23.587018    5580 scope.go:122] "RemoveContainer" containerID="416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"
Nov 02 22:15:23.587410 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:23.587144    5580 scope.go:122] "RemoveContainer" containerID="5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"
Nov 02 22:15:23.601262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:23.601219    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91a8e487-5b23-4a12-8cda-0c8e8988779d" path="/var/lib/kubelet/pods/91a8e487-5b23-4a12-8cda-0c8e8988779d/volumes"
Nov 02 22:15:24.470384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.470343    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 22:15:24.470603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.470581    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" containerID="cri-o://9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6" gracePeriod=30
Nov 02 22:15:24.480633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.480596    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 22:15:24.480851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.480824    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy" containerID="cri-o://252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f" gracePeriod=30
Nov 02 22:15:24.520225 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.520186    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerStarted","Data":"8eafc89c59a123328d94d3831282e19ef0d068d3d0f9b471e2e924a1d1f36bff"}
Nov 02 22:15:24.522503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.522476    5580 generic.go:356] "Generic (PLEG): container finished" podID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerID="746d6a84bacf83ebda7beb172d78f79a0f0007206d4f27ed3d687182c27425e1" exitCode=60
Nov 02 22:15:24.522647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.522557    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerDied","Data":"746d6a84bacf83ebda7beb172d78f79a0f0007206d4f27ed3d687182c27425e1"}
Nov 02 22:15:24.522647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.522614    5580 scope.go:122] "RemoveContainer" containerID="416c99f29cb7fb22c41c9233acb86d64afb6b12cc1a3e79a88e0671b6e3ae36f"
Nov 02 22:15:24.523042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.523027    5580 scope.go:122] "RemoveContainer" containerID="746d6a84bacf83ebda7beb172d78f79a0f0007206d4f27ed3d687182c27425e1"
Nov 02 22:15:24.523375 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:24.523355    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt_odh-ai-gateway-infra(b5c5327e-99aa-4432-b8d5-741b438e0929)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929"
Nov 02 22:15:24.524924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:24.524895    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerStarted","Data":"7d9e088499a4c7c737b43d590f7ae5076d839886c7688ee76f7cb6ea1baaa783"}
Nov 02 22:15:25.533536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.533502    5580 generic.go:356] "Generic (PLEG): container finished" podID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerID="8eafc89c59a123328d94d3831282e19ef0d068d3d0f9b471e2e924a1d1f36bff" exitCode=60
Nov 02 22:15:25.534114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.533579    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerDied","Data":"8eafc89c59a123328d94d3831282e19ef0d068d3d0f9b471e2e924a1d1f36bff"}
Nov 02 22:15:25.534114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.533703    5580 scope.go:122] "RemoveContainer" containerID="ccbf65e88cb4f20e58660d30ca3abaeee07ff61de777a53b137cb088a4725ca8"
Nov 02 22:15:25.534255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.534237    5580 scope.go:122] "RemoveContainer" containerID="8eafc89c59a123328d94d3831282e19ef0d068d3d0f9b471e2e924a1d1f36bff"
Nov 02 22:15:25.534602 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:25.534575    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29894295-9nplw_odh-ai-gateway-infra(885393f9-b131-4c1b-a4d1-0c9d110f6e7e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e"
Nov 02 22:15:25.537339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.537315    5580 generic.go:356] "Generic (PLEG): container finished" podID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerID="7d9e088499a4c7c737b43d590f7ae5076d839886c7688ee76f7cb6ea1baaa783" exitCode=60
Nov 02 22:15:25.537462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.537384    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerDied","Data":"7d9e088499a4c7c737b43d590f7ae5076d839886c7688ee76f7cb6ea1baaa783"}
Nov 02 22:15:25.537832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.537815    5580 scope.go:122] "RemoveContainer" containerID="7d9e088499a4c7c737b43d590f7ae5076d839886c7688ee76f7cb6ea1baaa783"
Nov 02 22:15:25.538152 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:25.538132    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf_odh-ai-gateway-infra(05352888-6b6c-4a3c-95ec-a3d83f40cd0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d"
Nov 02 22:15:25.547150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.547117    5580 scope.go:122] "RemoveContainer" containerID="5a447395fb9788d73988fb5a42fb860cbdc44998d1e86a78a02c3b93ae6b79c3"
Nov 02 22:15:25.641432 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.641393    5580 patch_prober.go:29] interesting pod/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:15:25.641617 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.641460    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:15:25.865039 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:25.857698    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:26.045977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.045218    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:26.114106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.114077    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:26.142358 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.142330    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:26.236445 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.236406    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\" (UniqueName: \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\") pod \"b5c5327e-99aa-4432-b8d5-741b438e0929\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") "
Nov 02 22:15:26.236682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.236661    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\") pod \"b5c5327e-99aa-4432-b8d5-741b438e0929\" (UID: \"b5c5327e-99aa-4432-b8d5-741b438e0929\") "
Nov 02 22:15:26.237009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.236984    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca" pod "b5c5327e-99aa-4432-b8d5-741b438e0929" (UID: "b5c5327e-99aa-4432-b8d5-741b438e0929"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:26.237209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.237195    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c5327e-99aa-4432-b8d5-741b438e0929-maas-api-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:26.238667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.238645    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h" pod "b5c5327e-99aa-4432-b8d5-741b438e0929" (UID: "b5c5327e-99aa-4432-b8d5-741b438e0929"). InnerVolumeSpecName "kube-api-access-svc7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:26.316501 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:26.316459    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:15:26.339015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.338975    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svc7h\" (UniqueName: \"kubernetes.io/projected/b5c5327e-99aa-4432-b8d5-741b438e0929-kube-api-access-svc7h\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:26.547348 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.547319    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"
Nov 02 22:15:26.547802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.547325    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt" event={"ID":"b5c5327e-99aa-4432-b8d5-741b438e0929","Type":"ContainerDied","Data":"2c55d224425da9ae4a874861c9daa355979bd6c0086e00e4fc4c3f7d9776c229"}
Nov 02 22:15:26.547802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.547419    5580 scope.go:122] "RemoveContainer" containerID="746d6a84bacf83ebda7beb172d78f79a0f0007206d4f27ed3d687182c27425e1"
Nov 02 22:15:26.719652 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.719615    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:26.731870 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.731843    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:26.743919 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.743877    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\" (UniqueName: \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\") pod \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") "
Nov 02 22:15:26.744092 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.744071    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\") pod \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\" (UID: \"885393f9-b131-4c1b-a4d1-0c9d110f6e7e\") "
Nov 02 22:15:26.744493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.744466    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca" pod "885393f9-b131-4c1b-a4d1-0c9d110f6e7e" (UID: "885393f9-b131-4c1b-a4d1-0c9d110f6e7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:26.744607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.744592    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-maas-api-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:26.746140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.746111    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7" pod "885393f9-b131-4c1b-a4d1-0c9d110f6e7e" (UID: "885393f9-b131-4c1b-a4d1-0c9d110f6e7e"). InnerVolumeSpecName "kube-api-access-4snl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:26.773686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.773644    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:26.778723 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.778687    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-267d89-29894295-bcbtt"]
Nov 02 22:15:26.845692 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.845583    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\" (UniqueName: \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\") pod \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") "
Nov 02 22:15:26.845899 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.845785    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\") pod \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\" (UID: \"05352888-6b6c-4a3c-95ec-a3d83f40cd0d\") "
Nov 02 22:15:26.846193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.846166    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca" pod "05352888-6b6c-4a3c-95ec-a3d83f40cd0d" (UID: "05352888-6b6c-4a3c-95ec-a3d83f40cd0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:26.846356 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.846340    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4snl7\" (UniqueName: \"kubernetes.io/projected/885393f9-b131-4c1b-a4d1-0c9d110f6e7e-kube-api-access-4snl7\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:26.846403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.846358    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-maas-api-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:26.847722 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.847698    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj" pod "05352888-6b6c-4a3c-95ec-a3d83f40cd0d" (UID: "05352888-6b6c-4a3c-95ec-a3d83f40cd0d"). InnerVolumeSpecName "kube-api-access-ck7bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:26.947822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:26.947782    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck7bj\" (UniqueName: \"kubernetes.io/projected/05352888-6b6c-4a3c-95ec-a3d83f40cd0d-kube-api-access-ck7bj\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:27.557620 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.557590    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"
Nov 02 22:15:27.558183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.557632    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw" event={"ID":"885393f9-b131-4c1b-a4d1-0c9d110f6e7e","Type":"ContainerDied","Data":"6b04fb548831d6d281ee520db1a6820e3267bca5c71cd20b3d20ad4796dbfcdd"}
Nov 02 22:15:27.558183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.557699    5580 scope.go:122] "RemoveContainer" containerID="8eafc89c59a123328d94d3831282e19ef0d068d3d0f9b471e2e924a1d1f36bff"
Nov 02 22:15:27.560760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.560729    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf" event={"ID":"05352888-6b6c-4a3c-95ec-a3d83f40cd0d","Type":"ContainerDied","Data":"3d8dfe1a12bc624d41567a237eb86cbbefb1a7c8af1b11b52b705fa4c125145e"}
Nov 02 22:15:27.560888 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.560840    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"
Nov 02 22:15:27.570796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.570769    5580 scope.go:122] "RemoveContainer" containerID="7d9e088499a4c7c737b43d590f7ae5076d839886c7688ee76f7cb6ea1baaa783"
Nov 02 22:15:27.595981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.595916    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" path="/var/lib/kubelet/pods/b5c5327e-99aa-4432-b8d5-741b438e0929/volumes"
Nov 02 22:15:27.719081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.719041    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:27.723175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.723140    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894295-9nplw"]
Nov 02 22:15:27.831420 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.831323    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:27.833662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:27.833638    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-af9e7c-29894295-s2pcf"]
Nov 02 22:15:29.594999 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.594949    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" path="/var/lib/kubelet/pods/05352888-6b6c-4a3c-95ec-a3d83f40cd0d/volumes"
Nov 02 22:15:29.595631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.595611    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" path="/var/lib/kubelet/pods/885393f9-b131-4c1b-a4d1-0c9d110f6e7e/volumes"
Nov 02 22:15:29.772211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.772171    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 22:15:29.810440 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.810401    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 22:15:29.886043 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886006    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886068    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886097    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886127    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886174    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886222    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886262    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886329    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886361    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886361    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886405    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886468    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886497    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886514 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886514    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886520    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886515    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886574    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886621    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886640    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886688    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\") pod \"c478823c-48ed-4e69-895c-705a33ad50ba\" (UID: \"c478823c-48ed-4e69-895c-705a33ad50ba\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886744    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886810    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886838    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.886977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.886853    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\") pod \"c07cfac2-2198-4366-a122-0085f921e52d\" (UID: \"c07cfac2-2198-4366-a122-0085f921e52d\") "
Nov 02 22:15:29.887479 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887175    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:29.887479 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887382    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.887637 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887612    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.887764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887748    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c478823c-48ed-4e69-895c-705a33ad50ba-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.887868 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887843    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.887868 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887870    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.888180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887884    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.888180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887899    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.888180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887913    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.888180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.887927    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.888180 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.888139    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.889388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889205    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:15:29.889388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889285    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "kube-api-access-6mpbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:29.889388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889322    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:29.889388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889353    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "kube-api-access-mvf54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:29.889709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889582    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:29.889709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889629    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.889791 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.889782    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:15:29.890030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.890002    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token" pod "c07cfac2-2198-4366-a122-0085f921e52d" (UID: "c07cfac2-2198-4366-a122-0085f921e52d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:29.890297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.890275    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy" pod "c478823c-48ed-4e69-895c-705a33ad50ba" (UID: "c478823c-48ed-4e69-895c-705a33ad50ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989149    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c478823c-48ed-4e69-895c-705a33ad50ba-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989177    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989193    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvf54\" (UniqueName: \"kubernetes.io/projected/c07cfac2-2198-4366-a122-0085f921e52d-kube-api-access-mvf54\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989207    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989217    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c07cfac2-2198-4366-a122-0085f921e52d-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989225    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mpbt\" (UniqueName: \"kubernetes.io/projected/c478823c-48ed-4e69-895c-705a33ad50ba-kube-api-access-6mpbt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989236    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989245    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c478823c-48ed-4e69-895c-705a33ad50ba-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989253    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c07cfac2-2198-4366-a122-0085f921e52d-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:29.989847 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:29.989261    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c07cfac2-2198-4366-a122-0085f921e52d-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:30.589421 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.589383    5580 generic.go:356] "Generic (PLEG): container finished" podID="c478823c-48ed-4e69-895c-705a33ad50ba" containerID="252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f" exitCode=0
Nov 02 22:15:30.589628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.589447    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"
Nov 02 22:15:30.589628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.589464    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" event={"ID":"c478823c-48ed-4e69-895c-705a33ad50ba","Type":"ContainerDied","Data":"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"}
Nov 02 22:15:30.589628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.589514    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt" event={"ID":"c478823c-48ed-4e69-895c-705a33ad50ba","Type":"ContainerDied","Data":"5e56d3587e6996fef0ff31bc98036e47f3facebd19ffa1a6efe04b1586e3079d"}
Nov 02 22:15:30.589628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.589542    5580 scope.go:122] "RemoveContainer" containerID="252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"
Nov 02 22:15:30.591074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.590929    5580 generic.go:356] "Generic (PLEG): container finished" podID="c07cfac2-2198-4366-a122-0085f921e52d" containerID="9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6" exitCode=0
Nov 02 22:15:30.591074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.590994    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"
Nov 02 22:15:30.591211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.590988    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" event={"ID":"c07cfac2-2198-4366-a122-0085f921e52d","Type":"ContainerDied","Data":"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"}
Nov 02 22:15:30.591211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.591125    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj" event={"ID":"c07cfac2-2198-4366-a122-0085f921e52d","Type":"ContainerDied","Data":"ec79810b55ac2f48ea450ccfb668824437cfc3aff9085fd5e4798070f27a2278"}
Nov 02 22:15:30.600396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.600373    5580 scope.go:122] "RemoveContainer" containerID="252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"
Nov 02 22:15:30.600765 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:30.600739    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f\": container with ID starting with 252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f not found: ID does not exist" containerID="252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"
Nov 02 22:15:30.600905 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.600766    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f"} err="failed to get container status \"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f\": rpc error: code = NotFound desc = could not find container \"252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f\": container with ID starting with 252bf16735199b8a6968abf26b3d8641d235859dd60dcae2627e5f0899e2d16f not found: ID does not exist"
Nov 02 22:15:30.600905 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.600791    5580 scope.go:122] "RemoveContainer" containerID="9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"
Nov 02 22:15:30.610951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.610925    5580 scope.go:122] "RemoveContainer" containerID="9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"
Nov 02 22:15:30.611326 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:30.611303    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6\": container with ID starting with 9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6 not found: ID does not exist" containerID="9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"
Nov 02 22:15:30.611389 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.611337    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6"} err="failed to get container status \"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6\": rpc error: code = NotFound desc = could not find container \"9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6\": container with ID starting with 9f7e5799fbf734edd5f03bf55f9592668d3a86dfaf00a9cf171dded1fc1ed3b6 not found: ID does not exist"
Nov 02 22:15:30.765345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.765256    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 22:15:30.768859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.768829    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-cjnwj"]
Nov 02 22:15:30.878021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.877975    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 22:15:30.882845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:30.882811    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-64438f-openshift-default-6996f8785-5zvrt"]
Nov 02 22:15:31.596403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:31.596368    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c07cfac2-2198-4366-a122-0085f921e52d" path="/var/lib/kubelet/pods/c07cfac2-2198-4366-a122-0085f921e52d/volumes"
Nov 02 22:15:31.596909 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:31.596894    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" path="/var/lib/kubelet/pods/c478823c-48ed-4e69-895c-705a33ad50ba/volumes"
Nov 02 22:15:33.102472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.102432    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:15:33.102951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.102666    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" podUID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" containerName="maas-api" containerID="cri-o://ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435" gracePeriod=30
Nov 02 22:15:33.401582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.401555    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:15:33.546768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.546732    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\") pod \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") "
Nov 02 22:15:33.546978 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.546853    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\") pod \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") "
Nov 02 22:15:33.547136 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.547111    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\" (UniqueName: \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\") pod \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") "
Nov 02 22:15:33.547279 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.547203    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") pod \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\" (UID: \"be91195f-5b0e-4bcb-a7ea-1b57192335ce\") "
Nov 02 22:15:33.547345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.547288    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca" pod "be91195f-5b0e-4bcb-a7ea-1b57192335ce" (UID: "be91195f-5b0e-4bcb-a7ea-1b57192335ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:15:33.547700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.547676    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be91195f-5b0e-4bcb-a7ea-1b57192335ce-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:33.549171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.549138    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs" pod "be91195f-5b0e-4bcb-a7ea-1b57192335ce" (UID: "be91195f-5b0e-4bcb-a7ea-1b57192335ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:15:33.549363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.549342    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs" pod "be91195f-5b0e-4bcb-a7ea-1b57192335ce" (UID: "be91195f-5b0e-4bcb-a7ea-1b57192335ce"). InnerVolumeSpecName "kube-api-access-s8dcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:33.549914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.549887    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls" pod "be91195f-5b0e-4bcb-a7ea-1b57192335ce" (UID: "be91195f-5b0e-4bcb-a7ea-1b57192335ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:15:33.615021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.614977    5580 generic.go:356] "Generic (PLEG): container finished" podID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" containerID="ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435" exitCode=0
Nov 02 22:15:33.615201 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.615026    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"
Nov 02 22:15:33.615201 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.615062    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" event={"ID":"be91195f-5b0e-4bcb-a7ea-1b57192335ce","Type":"ContainerDied","Data":"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"}
Nov 02 22:15:33.615201 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.615122    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk" event={"ID":"be91195f-5b0e-4bcb-a7ea-1b57192335ce","Type":"ContainerDied","Data":"9f3bafbd0554ca49e77c33ea3a5fda6d3d892616ca6d131c3271fd09f3ad3a52"}
Nov 02 22:15:33.615201 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.615145    5580 scope.go:122] "RemoveContainer" containerID="ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"
Nov 02 22:15:33.625635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.625613    5580 scope.go:122] "RemoveContainer" containerID="ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"
Nov 02 22:15:33.625997 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:33.625943    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435\": container with ID starting with ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435 not found: ID does not exist" containerID="ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"
Nov 02 22:15:33.626066 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.626007    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435"} err="failed to get container status \"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435\": rpc error: code = NotFound desc = could not find container \"ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435\": container with ID starting with ceb6c2289cccccd933f8a6284e09fd4cbc66c23ad6d9053fd24a6b6e0bf27435 not found: ID does not exist"
Nov 02 22:15:33.686030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.683400    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8dcs\" (UniqueName: \"kubernetes.io/projected/be91195f-5b0e-4bcb-a7ea-1b57192335ce-kube-api-access-s8dcs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:33.686030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.683439    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:33.686030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.683454    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be91195f-5b0e-4bcb-a7ea-1b57192335ce-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:15:33.798231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.798186    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:15:33.800798 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:33.800774    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-75577e-75b9877dc6-9lcqk"]
Nov 02 22:15:35.594428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:35.594384    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" path="/var/lib/kubelet/pods/be91195f-5b0e-4bcb-a7ea-1b57192335ce/volumes"
Nov 02 22:15:47.473875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:47.473832    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 22:15:47.523122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:47.523082    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 22:15:49.934268 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:15:49.934231    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:15:56.313090 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:15:56.313042    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:16:02.496780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.496709    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" containerID="cri-o://cbbec0bf5cbfc6c87f9c61328a525c1f5f625dd21ee5da50e10f21b1567fda09" gracePeriod=45
Nov 02 22:16:02.544287 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.544227    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main" containerID="cri-o://d5e6982298825ee89d70cc5c81c6baf8dcb7a1467a5d9dcf956a470dd51fb702" gracePeriod=45
Nov 02 22:16:02.820422 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.820382    5580 generic.go:356] "Generic (PLEG): container finished" podID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerID="d5e6982298825ee89d70cc5c81c6baf8dcb7a1467a5d9dcf956a470dd51fb702" exitCode=0
Nov 02 22:16:02.820699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.820535    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" event={"ID":"5843dfcc-2d55-4553-a53e-67e0de3b4b2f","Type":"ContainerDied","Data":"d5e6982298825ee89d70cc5c81c6baf8dcb7a1467a5d9dcf956a470dd51fb702"}
Nov 02 22:16:02.822375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.822338    5580 generic.go:356] "Generic (PLEG): container finished" podID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerID="cbbec0bf5cbfc6c87f9c61328a525c1f5f625dd21ee5da50e10f21b1567fda09" exitCode=0
Nov 02 22:16:02.822521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.822405    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" event={"ID":"e421aade-93e3-4df1-8f7d-d4201f44944c","Type":"ContainerDied","Data":"cbbec0bf5cbfc6c87f9c61328a525c1f5f625dd21ee5da50e10f21b1567fda09"}
Nov 02 22:16:02.822521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.822431    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" event={"ID":"e421aade-93e3-4df1-8f7d-d4201f44944c","Type":"ContainerDied","Data":"d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa"}
Nov 02 22:16:02.822521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.822443    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d549cf33d6f702b89bad0a4ca2a4062d29aab18a66bc66cc2eef4dcd709160aa"
Nov 02 22:16:02.841244 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.841216    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 22:16:02.857825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.857790    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 22:16:02.996251 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996215    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\" (UniqueName: \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.996431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996261    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\" (UniqueName: \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.996431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996315    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.996431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996336    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.996431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996394    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.996431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996426    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.996711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996484    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.996711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996570    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.996711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996606    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\" (UniqueName: \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.996711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996653    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.996711 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996663    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\") pod \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\" (UID: \"5843dfcc-2d55-4553-a53e-67e0de3b4b2f\") "
Nov 02 22:16:02.997042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996694    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.997042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996771    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.997042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996824    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\" (UniqueName: \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\") pod \"e421aade-93e3-4df1-8f7d-d4201f44944c\" (UID: \"e421aade-93e3-4df1-8f7d-d4201f44944c\") "
Nov 02 22:16:02.997042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.996936    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.997239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997151    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997361    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997531    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997554    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997570    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997584    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:02.997621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997599    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:02.998101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.997977    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.999186 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.999163    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.999351 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.999316    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:02.999429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.999321    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:16:02.999429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.999326    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "kube-api-access-6cgwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:16:02.999665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:02.999643    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh" pod "e421aade-93e3-4df1-8f7d-d4201f44944c" (UID: "e421aade-93e3-4df1-8f7d-d4201f44944c"). InnerVolumeSpecName "kube-api-access-fwbgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:16:03.000138 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.000025    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs" pod "5843dfcc-2d55-4553-a53e-67e0de3b4b2f" (UID: "5843dfcc-2d55-4553-a53e-67e0de3b4b2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:16:03.099193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099153    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cgwl\" (UniqueName: \"kubernetes.io/projected/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-kube-api-access-6cgwl\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099193    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e421aade-93e3-4df1-8f7d-d4201f44944c-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099210    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099222    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099233    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5843dfcc-2d55-4553-a53e-67e0de3b4b2f-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099243    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e421aade-93e3-4df1-8f7d-d4201f44944c-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.099403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.099255    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwbgh\" (UniqueName: \"kubernetes.io/projected/e421aade-93e3-4df1-8f7d-d4201f44944c-kube-api-access-fwbgh\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:03.741457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.741407    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:16:03.831630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.831546    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"
Nov 02 22:16:03.831630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.831563    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"
Nov 02 22:16:03.831630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.831588    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj" event={"ID":"5843dfcc-2d55-4553-a53e-67e0de3b4b2f","Type":"ContainerDied","Data":"dabbc79aa005ee5b65d39b40d8f8a20f2ff637c3eaad66984f3f21e0e84a631f"}
Nov 02 22:16:03.831915 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.831648    5580 scope.go:122] "RemoveContainer" containerID="d5e6982298825ee89d70cc5c81c6baf8dcb7a1467a5d9dcf956a470dd51fb702"
Nov 02 22:16:03.993822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.993785    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 22:16:03.996452 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:03.996426    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/premium-w5-96c052-kserve-7985896d76-w4pbv"]
Nov 02 22:16:04.150036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.149897    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 22:16:04.153628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.153592    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-96c052/simulator-w5-96c052-kserve-867d8868d9-qv7nj"]
Nov 02 22:16:04.432154 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.432116    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 22:16:04.432416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.432392    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" podUID="0353af85-1789-481a-8933-8d40027ceff9" containerName="maas-api" containerID="cri-o://3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b" gracePeriod=30
Nov 02 22:16:04.620339 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.620287    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:16:04.621227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621198    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main"
Nov 02 22:16:04.621227 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621226    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621249    5580 cpu_manager.go:414] "containerMap: removing container" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621257    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621269    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621278    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621288    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621296    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621312    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621319    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621331    5580 cpu_manager.go:414] "containerMap: removing container" podUID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" containerName="maas-api"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621339    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" containerName="maas-api"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621354    5580 cpu_manager.go:414] "containerMap: removing container" podUID="91a8e487-5b23-4a12-8cda-0c8e8988779d" containerName="maas-api"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621362    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91a8e487-5b23-4a12-8cda-0c8e8988779d" containerName="maas-api"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621388    5580 cpu_manager.go:414] "containerMap: removing container" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621396    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621424    5580 cpu_manager.go:414] "containerMap: removing container" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621431    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621447    5580 cpu_manager.go:414] "containerMap: removing container" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621454    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.621462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621470    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621478    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621508    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621517    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621659    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621674    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621688    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621697    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621709    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621721    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c07cfac2-2198-4366-a122-0085f921e52d" containerName="istio-proxy"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621734    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" containerName="main"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621744    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621753    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621761    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" containerName="main"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621771    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c478823c-48ed-4e69-895c-705a33ad50ba" containerName="istio-proxy"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621780    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="be91195f-5b0e-4bcb-a7ea-1b57192335ce" containerName="maas-api"
Nov 02 22:16:04.622501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.621803    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="91a8e487-5b23-4a12-8cda-0c8e8988779d" containerName="maas-api"
Nov 02 22:16:04.625799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.625771    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.689768 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.689675    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1789c6\"/\"test-model-1789c6-kserve-self-signed-certs\""
Nov 02 22:16:04.689933 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.689891    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1789c6\"/\"openshift-service-ca.crt\""
Nov 02 22:16:04.689998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.689973    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1789c6\"/\"kube-root-ca.crt\""
Nov 02 22:16:04.690198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.690173    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1789c6\"/\"default-dockercfg-g88x9\""
Nov 02 22:16:04.703184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.700132    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:16:04.719481 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.719446    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 22:16:04.720333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720306    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.720507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720387    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.720630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720599    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnt95\" (UniqueName: \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.720700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720668    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.720762 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720723    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.720862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.720847    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.829826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.829751    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") pod \"0353af85-1789-481a-8933-8d40027ceff9\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") "
Nov 02 22:16:04.830924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.830862    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\") pod \"0353af85-1789-481a-8933-8d40027ceff9\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") "
Nov 02 22:16:04.831242 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.831182    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\") pod \"0353af85-1789-481a-8933-8d40027ceff9\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") "
Nov 02 22:16:04.831611 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.831562    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\" (UniqueName: \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\") pod \"0353af85-1789-481a-8933-8d40027ceff9\" (UID: \"0353af85-1789-481a-8933-8d40027ceff9\") "
Nov 02 22:16:04.847526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.847485    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.847972 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.847894    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnt95\" (UniqueName: \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.848212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.848158    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.848367 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.848343    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.848604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.848555    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.848836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.848744    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.856076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.851720    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.856076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.851993    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.856858 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.856813    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.866145 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.865043    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca" pod "0353af85-1789-481a-8933-8d40027ceff9" (UID: "0353af85-1789-481a-8933-8d40027ceff9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:16:04.881102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.878977    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.881102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.880486    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls" pod "0353af85-1789-481a-8933-8d40027ceff9" (UID: "0353af85-1789-481a-8933-8d40027ceff9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:16:04.888467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.888221    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.888920 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.888633    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9" pod "0353af85-1789-481a-8933-8d40027ceff9" (UID: "0353af85-1789-481a-8933-8d40027ceff9"). InnerVolumeSpecName "kube-api-access-sfww9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:16:04.890733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.890673    5580 generic.go:356] "Generic (PLEG): container finished" podID="0353af85-1789-481a-8933-8d40027ceff9" containerID="3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b" exitCode=0
Nov 02 22:16:04.890860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.890830    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" event={"ID":"0353af85-1789-481a-8933-8d40027ceff9","Type":"ContainerDied","Data":"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"}
Nov 02 22:16:04.890914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.890885    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j" event={"ID":"0353af85-1789-481a-8933-8d40027ceff9","Type":"ContainerDied","Data":"08629525c34ff04e95b0275a4f0aed0d79e6f566ff4454f7945c61ef12d94f8c"}
Nov 02 22:16:04.890984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.890923    5580 scope.go:122] "RemoveContainer" containerID="3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"
Nov 02 22:16:04.891117 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.891100    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"
Nov 02 22:16:04.896542 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.896494    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs" pod "0353af85-1789-481a-8933-8d40027ceff9" (UID: "0353af85-1789-481a-8933-8d40027ceff9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:16:04.914153 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.914118    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnt95\" (UniqueName: \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\") pod \"test-model-1789c6-kserve-647df95d87-w4t7w\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") " pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:04.915885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.915858    5580 scope.go:122] "RemoveContainer" containerID="3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"
Nov 02 22:16:04.916350 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:04.916322    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b\": container with ID starting with 3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b not found: ID does not exist" containerID="3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"
Nov 02 22:16:04.916422 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.916365    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b"} err="failed to get container status \"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b\": rpc error: code = NotFound desc = could not find container \"3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b\": container with ID starting with 3ededcad5d838ee0be826a2fecc80a4ce6636f2a97e2da535a559121a7b4808b not found: ID does not exist"
Nov 02 22:16:04.950062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.949947    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:04.950062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.950003    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0353af85-1789-481a-8933-8d40027ceff9-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:04.950062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.950018    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfww9\" (UniqueName: \"kubernetes.io/projected/0353af85-1789-481a-8933-8d40027ceff9-kube-api-access-sfww9\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:04.950062 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.950034    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0353af85-1789-481a-8933-8d40027ceff9-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:04.993575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:04.993538    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:05.311456 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.311355    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 22:16:05.316713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.316672    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-ffe386-774994f48b-7m96j"]
Nov 02 22:16:05.423212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.423175    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:16:05.594196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.594109    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0353af85-1789-481a-8933-8d40027ceff9" path="/var/lib/kubelet/pods/0353af85-1789-481a-8933-8d40027ceff9/volumes"
Nov 02 22:16:05.594658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.594641    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5843dfcc-2d55-4553-a53e-67e0de3b4b2f" path="/var/lib/kubelet/pods/5843dfcc-2d55-4553-a53e-67e0de3b4b2f/volumes"
Nov 02 22:16:05.595099 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.595082    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e421aade-93e3-4df1-8f7d-d4201f44944c" path="/var/lib/kubelet/pods/e421aade-93e3-4df1-8f7d-d4201f44944c/volumes"
Nov 02 22:16:05.901986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.901844    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" event={"ID":"642d3830-840c-45ad-9f91-6553a5f3d03f","Type":"ContainerStarted","Data":"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"}
Nov 02 22:16:05.901986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.901912    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" event={"ID":"642d3830-840c-45ad-9f91-6553a5f3d03f","Type":"ContainerStarted","Data":"67c42123e5927be873e9cef6ae6a04f9aa6718bdc174430119913a1c99ef4fca"}
Nov 02 22:16:05.993681 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:05.993648    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:06.015927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:06.015863    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" podStartSLOduration=2.015847714 podStartE2EDuration="2.015847714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:16:06.014667127 +0000 UTC m=+2464.531162901" watchObservedRunningTime="2026-11-02 22:16:06.015847714 +0000 UTC m=+2464.532343484"
Nov 02 22:16:12.331279 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331240    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:16:12.331820 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331804    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:12.331862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331821    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:12.331862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331837    5580 cpu_manager.go:414] "containerMap: removing container" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:12.331862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331842    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:12.331862 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331861    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0353af85-1789-481a-8933-8d40027ceff9" containerName="maas-api"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331868    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0353af85-1789-481a-8933-8d40027ceff9" containerName="maas-api"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331889    5580 cpu_manager.go:414] "containerMap: removing container" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331894    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885393f9-b131-4c1b-a4d1-0c9d110f6e7e" containerName="cleanup"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.331998    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="05352888-6b6c-4a3c-95ec-a3d83f40cd0d" containerName="cleanup"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.332010    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0353af85-1789-481a-8933-8d40027ceff9" containerName="maas-api"
Nov 02 22:16:12.332040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.332017    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5c5327e-99aa-4432-b8d5-741b438e0929" containerName="cleanup"
Nov 02 22:16:12.336802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.336780    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.391016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.390977    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:16:12.438153 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.438111    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af9e7c\"/\"test-model-af9e7c-kserve-self-signed-certs\""
Nov 02 22:16:12.438326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.438186    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af9e7c\"/\"default-dockercfg-bgztb\""
Nov 02 22:16:12.440420 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.440393    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af9e7c\"/\"openshift-service-ca.crt\""
Nov 02 22:16:12.440420 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.440409    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af9e7c\"/\"kube-root-ca.crt\""
Nov 02 22:16:12.443096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443075    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl9jm\" (UniqueName: \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.443166 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443114    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.443166 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443136    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.443277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443212    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.443335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443318    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.443449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.443433    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544575    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl9jm\" (UniqueName: \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544626    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544646    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544667    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544720    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.544893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.544821    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.545137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.545066    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.545137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.545107    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.545262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.545217    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.547294 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.547265    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.547538 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.547519    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.601305 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.601217    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl9jm\" (UniqueName: \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\") pod \"test-model-af9e7c-kserve-5f5b6d957-5cn5l\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") " pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:12.654980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:12.654333    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:13.076239 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:13.076204    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:16:13.078930 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:16:13.078893    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod101972f2_0c0f_4f0a_9212_41c61f396fe5.slice/crio-5b5a5f43eb16abbf780546a3c660a10f3f82b8476823d7c0833d0ae7cdf00baf WatchSource:0}: Error finding container 5b5a5f43eb16abbf780546a3c660a10f3f82b8476823d7c0833d0ae7cdf00baf: Status 404 returned error can't find the container with id 5b5a5f43eb16abbf780546a3c660a10f3f82b8476823d7c0833d0ae7cdf00baf
Nov 02 22:16:13.631526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:13.631484    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 22:16:13.631992 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:13.631755    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy" containerID="cri-o://4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551" gracePeriod=30
Nov 02 22:16:13.954590 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:13.954552    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" event={"ID":"101972f2-0c0f-4f0a-9212-41c61f396fe5","Type":"ContainerStarted","Data":"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"}
Nov 02 22:16:13.954793 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:13.954605    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" event={"ID":"101972f2-0c0f-4f0a-9212-41c61f396fe5","Type":"ContainerStarted","Data":"5b5a5f43eb16abbf780546a3c660a10f3f82b8476823d7c0833d0ae7cdf00baf"}
Nov 02 22:16:14.069472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:14.069400    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" podStartSLOduration=2.069380406 podStartE2EDuration="2.069380406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:16:14.067520124 +0000 UTC m=+2472.584015901" watchObservedRunningTime="2026-11-02 22:16:14.069380406 +0000 UTC m=+2472.585876181"
Nov 02 22:16:14.658986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:14.658933    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:14.994671 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:14.994620    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:15.007816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:15.007784    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:15.979783 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:15.979741    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:16:18.901533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.901468    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 22:16:18.987643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.987610    5580 generic.go:356] "Generic (PLEG): container finished" podID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerID="4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551" exitCode=0
Nov 02 22:16:18.987860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.987661    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"
Nov 02 22:16:18.987860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.987697    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" event={"ID":"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5","Type":"ContainerDied","Data":"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"}
Nov 02 22:16:18.987860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.987771    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8" event={"ID":"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5","Type":"ContainerDied","Data":"81d8c87e0679a6454b76f800fb5e268794ce8c1ed955a1ba7e93f89b35e138c5"}
Nov 02 22:16:18.987860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.987805    5580 scope.go:122] "RemoveContainer" containerID="4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"
Nov 02 22:16:18.997552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.997529    5580 scope.go:122] "RemoveContainer" containerID="4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"
Nov 02 22:16:18.997885 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:18.997863    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551\": container with ID starting with 4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551 not found: ID does not exist" containerID="4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"
Nov 02 22:16:18.997994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:18.997899    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551"} err="failed to get container status \"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551\": rpc error: code = NotFound desc = could not find container \"4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551\": container with ID starting with 4ad58a6d7d4dea13fd46a4ff6fc311808d93aab416974c444538fbad54e4f551 not found: ID does not exist"
Nov 02 22:16:19.031585 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031551    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.031774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031610    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.031774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031645    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.031774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031724    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.031774 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031752    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.032037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031813    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.032037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.031985    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:19.032037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032022    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:19.032209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032019    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.032209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032134    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.032209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032181    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\") pod \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\" (UID: \"ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5\") "
Nov 02 22:16:19.032382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032321    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032538    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032893    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032919    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032935    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.032970    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.033125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.033040    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:19.034370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.034349    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:16:19.034526 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.034503    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:16:19.034619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.034605    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:16:19.034698 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.034639    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj" pod "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" (UID: "ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5"). InnerVolumeSpecName "kube-api-access-plgtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:16:19.134205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.134168    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.134205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.134199    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.134205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.134210    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plgtj\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-kube-api-access-plgtj\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.134456 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.134219    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.134456 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.134228    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:16:19.407218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.407182    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 22:16:19.457096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.457058    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ffe386-openshift-default-756448cc4c-fgst8"]
Nov 02 22:16:19.594433 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:19.594344    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" path="/var/lib/kubelet/pods/ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5/volumes"
Nov 02 22:16:20.000415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:20.000375    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:16:22.658570 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:22.658530    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:22.671789 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:22.671758    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:23.027225 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:23.027189    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:16:26.311608 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:26.311569    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:16:34.532387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.532352    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:16:34.532933 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.532917    5580 cpu_manager.go:414] "containerMap: removing container" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy"
Nov 02 22:16:34.533016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.532936    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy"
Nov 02 22:16:34.533058 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.533046    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffc18d5c-9ec6-4ddf-9ab7-4e892f4bdfb5" containerName="istio-proxy"
Nov 02 22:16:34.536564 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.536544    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.639059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.639028    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-94a8a3-serving-cert\""
Nov 02 22:16:34.641454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.641431    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:16:34.669761 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.669717    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.669987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.669873    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsbwt\" (UniqueName: \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.669987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.669927    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.670255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.670227    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.772208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.772154    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.772430 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.772270    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.772430 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.772343    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsbwt\" (UniqueName: \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.772430 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.772388    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.775400 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:34.775253    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-serving-cert: secret "maas-api-e2e-inf-b-94a8a3-serving-cert" not found
Nov 02 22:16:34.775400 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:34.775345    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls podName:0c11894c-5c60-43e9-971e-8533012685c2 nodeName:}" failed. No retries permitted until 2026-11-02 22:16:35.275322675 +0000 UTC m=+2493.791818429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls") pod "maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" (UID: "0c11894c-5c60-43e9-971e-8533012685c2") : secret "maas-api-e2e-inf-b-94a8a3-serving-cert" not found
Nov 02 22:16:34.788552 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.788430    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.798198 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.797362    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:34.833694 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:34.833657    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsbwt\" (UniqueName: \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:35.279612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:35.279569    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:35.282272 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:35.282240    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") pod \"maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:35.451672 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:35.451630    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:35.827974 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:35.827934    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:16:35.830424 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:16:35.830387    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c11894c_5c60_43e9_971e_8533012685c2.slice/crio-a5b03ec40c18066548265b881c9f6baf090b88e7d3e7fde70a27a89773734a12 WatchSource:0}: Error finding container a5b03ec40c18066548265b881c9f6baf090b88e7d3e7fde70a27a89773734a12: Status 404 returned error can't find the container with id a5b03ec40c18066548265b881c9f6baf090b88e7d3e7fde70a27a89773734a12
Nov 02 22:16:36.119602 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:36.119502    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" event={"ID":"0c11894c-5c60-43e9-971e-8533012685c2","Type":"ContainerStarted","Data":"a5b03ec40c18066548265b881c9f6baf090b88e7d3e7fde70a27a89773734a12"}
Nov 02 22:16:37.129125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:37.129001    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" event={"ID":"0c11894c-5c60-43e9-971e-8533012685c2","Type":"ContainerStarted","Data":"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"}
Nov 02 22:16:37.129674 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:37.129183    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:37.243238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:37.243161    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" podStartSLOduration=2.3346458869999998 podStartE2EDuration="3.243145481s" totalImagesPullingTime="908.499594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:16:37.241683974 +0000 UTC m=+2495.758179750" watchObservedRunningTime="2026-11-02 22:16:37.243145481 +0000 UTC m=+2495.759641267"
Nov 02 22:16:43.141457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:43.141424    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:16:51.478389 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:16:51.478353    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:16:56.312198 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:16:56.312157    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:17:02.685350 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:02.685315    5580 scope.go:122] "RemoveContainer" containerID="cbbec0bf5cbfc6c87f9c61328a525c1f5f625dd21ee5da50e10f21b1567fda09"
Nov 02 22:17:17.074057 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:17.073950    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:17:26.311619 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:17:26.311573    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:17:30.694159 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.694109    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:17:30.698177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.698149    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.751708 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.751446    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d1a155\"/\"kube-root-ca.crt\""
Nov 02 22:17:30.792468 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792419    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.792689 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792589    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hfs\" (UniqueName: \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.792689 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792651    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.792781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792762    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.792820 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792791    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.792856 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.792839    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.800411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.800374    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d1a155\"/\"openshift-service-ca.crt\""
Nov 02 22:17:30.800611 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.800480    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d1a155\"/\"default-dockercfg-7p4k9\""
Nov 02 22:17:30.800611 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.800497    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d1a155\"/\"test-model-d1a155-kserve-self-signed-certs\""
Nov 02 22:17:30.802877 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.802852    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:17:30.893712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.893672    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hfs\" (UniqueName: \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.893924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.893751    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.893924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.893815    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.893924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.893840    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.893924 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.893884    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.894186 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.894033    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.894373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.894347    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.894437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.894416    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.894485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.894442    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.896743 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.896721    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.897096 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.897074    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:30.950462 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:30.950380    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hfs\" (UniqueName: \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\") pod \"test-model-d1a155-kserve-58f8688fd5-flgbp\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") " pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:31.010628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.010582    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:31.314175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.314095    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 22:17:31.314334 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.314300    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" podUID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" containerName="maas-api" containerID="cri-o://490407b28aa6e27b9170308b5010eb451eb1f978b5d2a2bb8ad8ffd4fe5e485b" gracePeriod=30
Nov 02 22:17:31.424765 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.424734    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:17:31.426386 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:17:31.426347    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda537edb1_4a8e_42d7_bd82_3ca736aa1969.slice/crio-06989eac345aa1f6f0f511737526796175de30bc2c285de90c67cd546030f463 WatchSource:0}: Error finding container 06989eac345aa1f6f0f511737526796175de30bc2c285de90c67cd546030f463: Status 404 returned error can't find the container with id 06989eac345aa1f6f0f511737526796175de30bc2c285de90c67cd546030f463
Nov 02 22:17:31.485647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.485613    5580 generic.go:356] "Generic (PLEG): container finished" podID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" containerID="490407b28aa6e27b9170308b5010eb451eb1f978b5d2a2bb8ad8ffd4fe5e485b" exitCode=0
Nov 02 22:17:31.485844 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.485665    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" event={"ID":"65528f8c-552e-47dc-af4c-6ac6c8d5324f","Type":"ContainerDied","Data":"490407b28aa6e27b9170308b5010eb451eb1f978b5d2a2bb8ad8ffd4fe5e485b"}
Nov 02 22:17:31.487497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.487467    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" event={"ID":"a537edb1-4a8e-42d7-bd82-3ca736aa1969","Type":"ContainerStarted","Data":"06989eac345aa1f6f0f511737526796175de30bc2c285de90c67cd546030f463"}
Nov 02 22:17:31.592326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.592297    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 22:17:31.714937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.709172    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") pod \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") "
Nov 02 22:17:31.714937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.709273    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\") pod \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") "
Nov 02 22:17:31.714937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.709359    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\" (UniqueName: \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\") pod \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") "
Nov 02 22:17:31.714937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.709400    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\") pod \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\" (UID: \"65528f8c-552e-47dc-af4c-6ac6c8d5324f\") "
Nov 02 22:17:31.727704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.727383    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca" pod "65528f8c-552e-47dc-af4c-6ac6c8d5324f" (UID: "65528f8c-552e-47dc-af4c-6ac6c8d5324f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:17:31.729086 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.729056    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs" pod "65528f8c-552e-47dc-af4c-6ac6c8d5324f" (UID: "65528f8c-552e-47dc-af4c-6ac6c8d5324f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:17:31.731596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.731095    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls" pod "65528f8c-552e-47dc-af4c-6ac6c8d5324f" (UID: "65528f8c-552e-47dc-af4c-6ac6c8d5324f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:17:31.731596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.731285    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw" pod "65528f8c-552e-47dc-af4c-6ac6c8d5324f" (UID: "65528f8c-552e-47dc-af4c-6ac6c8d5324f"). InnerVolumeSpecName "kube-api-access-xzmqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:17:31.811391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.811354    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:17:31.811391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.811386    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65528f8c-552e-47dc-af4c-6ac6c8d5324f-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:17:31.811391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.811396    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzmqw\" (UniqueName: \"kubernetes.io/projected/65528f8c-552e-47dc-af4c-6ac6c8d5324f-kube-api-access-xzmqw\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:17:31.811650 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:31.811407    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65528f8c-552e-47dc-af4c-6ac6c8d5324f-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:17:32.496040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.496007    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"
Nov 02 22:17:32.496259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.495999    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht" event={"ID":"65528f8c-552e-47dc-af4c-6ac6c8d5324f","Type":"ContainerDied","Data":"bbb965229ced19341a1801fb7d976181422e5b634f3eec6b4c9f611a02c384b4"}
Nov 02 22:17:32.496259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.496169    5580 scope.go:122] "RemoveContainer" containerID="490407b28aa6e27b9170308b5010eb451eb1f978b5d2a2bb8ad8ffd4fe5e485b"
Nov 02 22:17:32.498083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.498048    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" event={"ID":"a537edb1-4a8e-42d7-bd82-3ca736aa1969","Type":"ContainerStarted","Data":"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"}
Nov 02 22:17:32.658175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.658132    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 22:17:32.707928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.707887    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-96c052-99b4b85c9-bmqht"]
Nov 02 22:17:32.769757 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:32.769649    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" podStartSLOduration=2.769632498 podStartE2EDuration="2.769632498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:32.768716621 +0000 UTC m=+2551.285212396" watchObservedRunningTime="2026-11-02 22:17:32.769632498 +0000 UTC m=+2551.286128273"
Nov 02 22:17:33.011040 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:33.010989    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:33.594184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:33.594150    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" path="/var/lib/kubelet/pods/65528f8c-552e-47dc-af4c-6ac6c8d5324f/volumes"
Nov 02 22:17:39.546413 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:39.546371    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:17:41.011033 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:41.011001    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:41.027364 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:41.027333    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:41.566167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:17:41.566131    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:17:56.311768 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:17:56.311725    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:18:10.308037 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:18:10.307981    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:18:26.312138 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:18:26.312094    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:18:30.781558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:18:30.781521    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:18:49.311710 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:18:49.311668    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:18:54.469642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:18:54.469598    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:18:56.312633 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:18:56.312591    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:19:04.332971 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.332899    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" podUID="642d3830-840c-45ad-9f91-6553a5f3d03f" containerName="main" containerID="cri-o://cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24" gracePeriod=45
Nov 02 22:19:04.610666 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.610636    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:19:04.729813 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.729770    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.729905    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.729978    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\" (UniqueName: \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730055 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730009    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730067    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730091    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:04.730253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730149    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\" (UniqueName: \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\") pod \"642d3830-840c-45ad-9f91-6553a5f3d03f\" (UID: \"642d3830-840c-45ad-9f91-6553a5f3d03f\") "
Nov 02 22:19:04.730253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730207    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:04.730810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730786    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:04.730810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.730811    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:04.731105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.731084    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:04.733296 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.733266    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:19:04.733417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.733341    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "kube-api-access-rnt95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:19:04.734499 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.734471    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm" pod "642d3830-840c-45ad-9f91-6553a5f3d03f" (UID: "642d3830-840c-45ad-9f91-6553a5f3d03f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:04.832199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.832163    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:04.832199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.832196    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnt95\" (UniqueName: \"kubernetes.io/projected/642d3830-840c-45ad-9f91-6553a5f3d03f-kube-api-access-rnt95\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:04.832199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.832206    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/642d3830-840c-45ad-9f91-6553a5f3d03f-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:04.832451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:04.832216    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/642d3830-840c-45ad-9f91-6553a5f3d03f-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:05.095141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.095106    5580 generic.go:356] "Generic (PLEG): container finished" podID="642d3830-840c-45ad-9f91-6553a5f3d03f" containerID="cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24" exitCode=0
Nov 02 22:19:05.095326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.095154    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"
Nov 02 22:19:05.095326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.095172    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" event={"ID":"642d3830-840c-45ad-9f91-6553a5f3d03f","Type":"ContainerDied","Data":"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"}
Nov 02 22:19:05.095326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.095206    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w" event={"ID":"642d3830-840c-45ad-9f91-6553a5f3d03f","Type":"ContainerDied","Data":"67c42123e5927be873e9cef6ae6a04f9aa6718bdc174430119913a1c99ef4fca"}
Nov 02 22:19:05.095326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.095223    5580 scope.go:122] "RemoveContainer" containerID="cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"
Nov 02 22:19:05.106434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.106413    5580 scope.go:122] "RemoveContainer" containerID="cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"
Nov 02 22:19:05.106889 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:05.106787    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24\": container with ID starting with cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24 not found: ID does not exist" containerID="cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"
Nov 02 22:19:05.106889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.106851    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24"} err="failed to get container status \"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24\": rpc error: code = NotFound desc = could not find container \"cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24\": container with ID starting with cc04b2d864c0da759851f1db1be4bc7862b8294843f442fea17d0e0982528c24 not found: ID does not exist"
Nov 02 22:19:05.264813 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.260210    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:19:05.267653 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.267620    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1789c6/test-model-1789c6-kserve-647df95d87-w4t7w"]
Nov 02 22:19:05.594226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:05.594192    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="642d3830-840c-45ad-9f91-6553a5f3d03f" path="/var/lib/kubelet/pods/642d3830-840c-45ad-9f91-6553a5f3d03f/volumes"
Nov 02 22:19:24.322939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:24.322896    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:19:25.292376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:25.292342    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:19:26.310455 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:26.310414    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:19:40.313515 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.313466    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" podUID="101972f2-0c0f-4f0a-9212-41c61f396fe5" containerName="main" containerID="cri-o://5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742" gracePeriod=45
Nov 02 22:19:40.584771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.584695    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:19:40.596383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596344    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\" (UniqueName: \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596437    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596681 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596634    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596681 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596669    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596712    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596792 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596753    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:40.596912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596831    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\" (UniqueName: \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\") pod \"101972f2-0c0f-4f0a-9212-41c61f396fe5\" (UID: \"101972f2-0c0f-4f0a-9212-41c61f396fe5\") "
Nov 02 22:19:40.596983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596920    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:40.596983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.596975    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:40.597416 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.597390    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.597589 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.597575    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.597700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.597679    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.599330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.599301    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "kube-api-access-gl9jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:19:40.599544 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.599517    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:19:40.599628 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.599606    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm" pod "101972f2-0c0f-4f0a-9212-41c61f396fe5" (UID: "101972f2-0c0f-4f0a-9212-41c61f396fe5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:40.688486 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.688449    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:19:40.688692 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.688655    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" podUID="705e02c7-d567-4802-ae53-a1a83f630d9e" containerName="maas-api" containerID="cri-o://a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77" gracePeriod=30
Nov 02 22:19:40.698710 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.698673    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/101972f2-0c0f-4f0a-9212-41c61f396fe5-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.698710 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.698709    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl9jm\" (UniqueName: \"kubernetes.io/projected/101972f2-0c0f-4f0a-9212-41c61f396fe5-kube-api-access-gl9jm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.698895 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.698733    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/101972f2-0c0f-4f0a-9212-41c61f396fe5-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:40.969005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:40.968976    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:19:41.003593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.003559    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\") pod \"705e02c7-d567-4802-ae53-a1a83f630d9e\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") "
Nov 02 22:19:41.003800 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.003636    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") pod \"705e02c7-d567-4802-ae53-a1a83f630d9e\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") "
Nov 02 22:19:41.003800 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.003782    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\") pod \"705e02c7-d567-4802-ae53-a1a83f630d9e\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") "
Nov 02 22:19:41.003925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.003828    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\" (UniqueName: \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\") pod \"705e02c7-d567-4802-ae53-a1a83f630d9e\" (UID: \"705e02c7-d567-4802-ae53-a1a83f630d9e\") "
Nov 02 22:19:41.004173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.004139    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca" pod "705e02c7-d567-4802-ae53-a1a83f630d9e" (UID: "705e02c7-d567-4802-ae53-a1a83f630d9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:19:41.004330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.004311    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/705e02c7-d567-4802-ae53-a1a83f630d9e-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:41.005912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.005881    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs" pod "705e02c7-d567-4802-ae53-a1a83f630d9e" (UID: "705e02c7-d567-4802-ae53-a1a83f630d9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:19:41.005912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.005893    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7" pod "705e02c7-d567-4802-ae53-a1a83f630d9e" (UID: "705e02c7-d567-4802-ae53-a1a83f630d9e"). InnerVolumeSpecName "kube-api-access-qskx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:19:41.005912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.005902    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls" pod "705e02c7-d567-4802-ae53-a1a83f630d9e" (UID: "705e02c7-d567-4802-ae53-a1a83f630d9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:19:41.105858 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.105817    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qskx7\" (UniqueName: \"kubernetes.io/projected/705e02c7-d567-4802-ae53-a1a83f630d9e-kube-api-access-qskx7\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:41.105858 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.105857    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:41.106100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.105872    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/705e02c7-d567-4802-ae53-a1a83f630d9e-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:41.309382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.309347    5580 generic.go:356] "Generic (PLEG): container finished" podID="705e02c7-d567-4802-ae53-a1a83f630d9e" containerID="a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77" exitCode=0
Nov 02 22:19:41.309576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.309399    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"
Nov 02 22:19:41.309576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.309429    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" event={"ID":"705e02c7-d567-4802-ae53-a1a83f630d9e","Type":"ContainerDied","Data":"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"}
Nov 02 22:19:41.309576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.309484    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55" event={"ID":"705e02c7-d567-4802-ae53-a1a83f630d9e","Type":"ContainerDied","Data":"a2fb4a6a376514957f5d5c67affa724d6e9fdcf86806f4b46f8423c05c3991fb"}
Nov 02 22:19:41.309576 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.309515    5580 scope.go:122] "RemoveContainer" containerID="a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"
Nov 02 22:19:41.311245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.311218    5580 generic.go:356] "Generic (PLEG): container finished" podID="101972f2-0c0f-4f0a-9212-41c61f396fe5" containerID="5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742" exitCode=0
Nov 02 22:19:41.311369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.311257    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"
Nov 02 22:19:41.311369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.311306    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" event={"ID":"101972f2-0c0f-4f0a-9212-41c61f396fe5","Type":"ContainerDied","Data":"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"}
Nov 02 22:19:41.311369 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.311355    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l" event={"ID":"101972f2-0c0f-4f0a-9212-41c61f396fe5","Type":"ContainerDied","Data":"5b5a5f43eb16abbf780546a3c660a10f3f82b8476823d7c0833d0ae7cdf00baf"}
Nov 02 22:19:41.321291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.321095    5580 scope.go:122] "RemoveContainer" containerID="a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"
Nov 02 22:19:41.321543 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:41.321379    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77\": container with ID starting with a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77 not found: ID does not exist" containerID="a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"
Nov 02 22:19:41.321543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.321401    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77"} err="failed to get container status \"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77\": rpc error: code = NotFound desc = could not find container \"a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77\": container with ID starting with a41fa2271f914892550f036c402b7129f41d59be5ac98b381406f2db6c0dae77 not found: ID does not exist"
Nov 02 22:19:41.321543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.321417    5580 scope.go:122] "RemoveContainer" containerID="5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"
Nov 02 22:19:41.341914 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.341892    5580 scope.go:122] "RemoveContainer" containerID="5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"
Nov 02 22:19:41.342274 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:41.342258    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742\": container with ID starting with 5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742 not found: ID does not exist" containerID="5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"
Nov 02 22:19:41.342331 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.342282    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742"} err="failed to get container status \"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742\": rpc error: code = NotFound desc = could not find container \"5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742\": container with ID starting with 5f9c9ac2246b6df26dbae8417144b7f4bda62111206e42679c475203d8c78742 not found: ID does not exist"
Nov 02 22:19:41.476912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.476872    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:19:41.525853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.525813    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1789c6-5d9696845d-4rb55"]
Nov 02 22:19:41.593905 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.593782    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="705e02c7-d567-4802-ae53-a1a83f630d9e" path="/var/lib/kubelet/pods/705e02c7-d567-4802-ae53-a1a83f630d9e/volumes"
Nov 02 22:19:41.615625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.615591    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:19:41.618618 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:41.618580    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-af9e7c/test-model-af9e7c-kserve-5f5b6d957-5cn5l"]
Nov 02 22:19:43.593222 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:43.593188    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="101972f2-0c0f-4f0a-9212-41c61f396fe5" path="/var/lib/kubelet/pods/101972f2-0c0f-4f0a-9212-41c61f396fe5/volumes"
Nov 02 22:19:49.733373 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:49.733332    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:19:49.939715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:49.939680    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:19:49.939936 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:49.939908    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" containerID="cri-o://f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed" gracePeriod=30
Nov 02 22:19:52.200889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:52.200849    5580 patch_prober.go:29] interesting pod/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:19:52.201315 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:52.200929    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:19:55.213824 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.213761    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:19:55.307511 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307468    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307528    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307594    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307634    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307685    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307717    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307796    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307883    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.307977 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.307923    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\") pod \"83d29e7f-e347-453d-8e46-abdc72da06b2\" (UID: \"83d29e7f-e347-453d-8e46-abdc72da06b2\") "
Nov 02 22:19:55.308262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308055    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:55.308262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308163    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:55.308262 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308088    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:19:55.308414 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308303    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:55.308465 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308407    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:55.308698 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308675    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.308772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308704    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.308772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308720    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.308772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308734    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.308772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.308748    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83d29e7f-e347-453d-8e46-abdc72da06b2-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.310315 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.310285    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:19:55.310458 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.310328    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:19:55.310518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.310492    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:19:55.310631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.310608    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd" pod "83d29e7f-e347-453d-8e46-abdc72da06b2" (UID: "83d29e7f-e347-453d-8e46-abdc72da06b2"). InnerVolumeSpecName "kube-api-access-mdprd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:19:55.399931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.399892    5580 generic.go:356] "Generic (PLEG): container finished" podID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerID="f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed" exitCode=0
Nov 02 22:19:55.400197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.399950    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"
Nov 02 22:19:55.400197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.399997    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerDied","Data":"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"}
Nov 02 22:19:55.400197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.400069    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7" event={"ID":"83d29e7f-e347-453d-8e46-abdc72da06b2","Type":"ContainerDied","Data":"cb35e74ca024caf8d4b1e67d994cd3bc4a691b0aa9387147494c075b5f156cb6"}
Nov 02 22:19:55.400197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.400102    5580 scope.go:122] "RemoveContainer" containerID="f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"
Nov 02 22:19:55.412762 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.412720    5580 scope.go:122] "RemoveContainer" containerID="4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"
Nov 02 22:19:55.414980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.414804    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.414980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.414845    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdprd\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-kube-api-access-mdprd\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.414980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.414860    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.414980 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.414872    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83d29e7f-e347-453d-8e46-abdc72da06b2-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:19:55.431704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.431674    5580 scope.go:122] "RemoveContainer" containerID="f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"
Nov 02 22:19:55.432130 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:55.432104    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed\": container with ID starting with f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed not found: ID does not exist" containerID="f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"
Nov 02 22:19:55.432224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.432141    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed"} err="failed to get container status \"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed\": rpc error: code = NotFound desc = could not find container \"f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed\": container with ID starting with f62416d1a3e128eb3013438be62cba618b1e6e72738de79aaa477560b73c55ed not found: ID does not exist"
Nov 02 22:19:55.432224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.432164    5580 scope.go:122] "RemoveContainer" containerID="4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"
Nov 02 22:19:55.432582 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:55.432554    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a\": container with ID starting with 4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a not found: ID does not exist" containerID="4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"
Nov 02 22:19:55.432682 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.432592    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a"} err="failed to get container status \"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a\": rpc error: code = NotFound desc = could not find container \"4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a\": container with ID starting with 4269f3644b5936a82130fa454305fe4d5ada967465c28dcbc9639ea594f4a10a not found: ID does not exist"
Nov 02 22:19:55.548175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.548073    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:19:55.552002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.551971    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1789c6-openshift-default-75f7dbf855-lqgb7"]
Nov 02 22:19:55.593391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:19:55.593349    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" path="/var/lib/kubelet/pods/83d29e7f-e347-453d-8e46-abdc72da06b2/volumes"
Nov 02 22:19:56.312684 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:19:56.312641    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:20:02.221515 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.221484    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:20:02.230014 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.229986    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:20:02.232335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.232312    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:20:02.240295 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.240274    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:20:02.426806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.426764    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:20:02.536451 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:02.536373    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:20:07.427704 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.427635    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" podUID="12236a2c-5fca-46cd-aaa7-f05e01670f06" containerName="payload-processing" containerID="cri-o://0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f" gracePeriod=25
Nov 02 22:20:07.537471 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.537417    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing" containerID="cri-o://bfcce6809f2fff02cc8f2b28a9390d8fe42457f61b28e46d0796359ca675acf9" gracePeriod=25
Nov 02 22:20:07.696997 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.696911    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:20:07.774925 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.774887    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\" (UniqueName: \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\") pod \"12236a2c-5fca-46cd-aaa7-f05e01670f06\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") "
Nov 02 22:20:07.775139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.775009    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\") pod \"12236a2c-5fca-46cd-aaa7-f05e01670f06\" (UID: \"12236a2c-5fca-46cd-aaa7-f05e01670f06\") "
Nov 02 22:20:07.775395 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.775366    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume" pod "12236a2c-5fca-46cd-aaa7-f05e01670f06" (UID: "12236a2c-5fca-46cd-aaa7-f05e01670f06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:07.777213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.777186    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq" pod "12236a2c-5fca-46cd-aaa7-f05e01670f06" (UID: "12236a2c-5fca-46cd-aaa7-f05e01670f06"). InnerVolumeSpecName "kube-api-access-f6jpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:07.876255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.876218    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6jpq\" (UniqueName: \"kubernetes.io/projected/12236a2c-5fca-46cd-aaa7-f05e01670f06-kube-api-access-f6jpq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:07.876255 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:07.876246    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12236a2c-5fca-46cd-aaa7-f05e01670f06-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:08.478164 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.478128    5580 generic.go:356] "Generic (PLEG): container finished" podID="12236a2c-5fca-46cd-aaa7-f05e01670f06" containerID="0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f" exitCode=0
Nov 02 22:20:08.478688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.478189    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"
Nov 02 22:20:08.478688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.478213    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" event={"ID":"12236a2c-5fca-46cd-aaa7-f05e01670f06","Type":"ContainerDied","Data":"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"}
Nov 02 22:20:08.478688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.478268    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq" event={"ID":"12236a2c-5fca-46cd-aaa7-f05e01670f06","Type":"ContainerDied","Data":"c4b1e5f64254e53231285000f1384d2eb4e389932a78f57212dc19f374ad284e"}
Nov 02 22:20:08.478688 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.478292    5580 scope.go:122] "RemoveContainer" containerID="0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"
Nov 02 22:20:08.488410 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.488389    5580 scope.go:122] "RemoveContainer" containerID="0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"
Nov 02 22:20:08.488744 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:08.488722    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f\": container with ID starting with 0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f not found: ID does not exist" containerID="0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"
Nov 02 22:20:08.488817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.488758    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f"} err="failed to get container status \"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f\": rpc error: code = NotFound desc = could not find container \"0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f\": container with ID starting with 0ce19e13e9de3d95aceb94e92d57c09cec1e2a80409cb4f41e53c380108c2d1f not found: ID does not exist"
Nov 02 22:20:08.648020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.647973    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:20:08.653973 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:08.653930    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af9e7c-58d889bb57-xjglq"]
Nov 02 22:20:09.593831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:09.593791    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12236a2c-5fca-46cd-aaa7-f05e01670f06" path="/var/lib/kubelet/pods/12236a2c-5fca-46cd-aaa7-f05e01670f06/volumes"
Nov 02 22:20:13.568703 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:13.568624    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:20:15.434363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:15.434320    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:20:18.314274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.314234    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 22:20:18.314712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.314452    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" containerID="cri-o://2a7214109965d944754327e2089097f5768cf80d539a7d8d13b8914c387ebd12" gracePeriod=30
Nov 02 22:20:18.369083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.369040    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:20:18.369311 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.369286    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy" containerID="cri-o://ac34eeec4ff75bef7b5737fdf4b0c007123d5d5b790b9d0ebfa4028645634870" gracePeriod=30
Nov 02 22:20:18.419307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.419260    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:20:18.419571 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.419546    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" containerID="cri-o://495acdffc99b5dbf3bf1bb8cc321d543f0429aa8968afc127ee0ff39cb91c24c" gracePeriod=30
Nov 02 22:20:18.421308 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.421284    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:20:18.421516 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.421491    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy" containerID="cri-o://e0b0d91945c504e9f104cbfe4b0e4d916846ad6dc4244e0d56c721a9dd0b478a" gracePeriod=30
Nov 02 22:20:18.432431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.432386    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:20:18.432740 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:18.432714    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" containerID="cri-o://5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d" gracePeriod=30
Nov 02 22:20:21.246497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:21.246457    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-kq8f9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:20:21.246984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:21.246515    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:20:21.331197 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:21.331157    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-24z5d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:20:21.331380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:21.331213    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:20:22.149211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:22.149170    5580 patch_prober.go:29] interesting pod/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:20:22.149419 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:22.149244    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:20:23.605381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.591830    5580 generic.go:356] "Generic (PLEG): container finished" podID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerID="2a7214109965d944754327e2089097f5768cf80d539a7d8d13b8914c387ebd12" exitCode=0
Nov 02 22:20:23.605381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.595389    5580 generic.go:356] "Generic (PLEG): container finished" podID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerID="ac34eeec4ff75bef7b5737fdf4b0c007123d5d5b790b9d0ebfa4028645634870" exitCode=0
Nov 02 22:20:23.605381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.599631    5580 generic.go:356] "Generic (PLEG): container finished" podID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerID="495acdffc99b5dbf3bf1bb8cc321d543f0429aa8968afc127ee0ff39cb91c24c" exitCode=0
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.610479    5580 generic.go:356] "Generic (PLEG): container finished" podID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerID="e0b0d91945c504e9f104cbfe4b0e4d916846ad6dc4244e0d56c721a9dd0b478a" exitCode=0
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.614354    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerDied","Data":"2a7214109965d944754327e2089097f5768cf80d539a7d8d13b8914c387ebd12"}
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.614422    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" event={"ID":"74d46a2f-20e1-4b5b-b169-e8610095770f","Type":"ContainerDied","Data":"ac34eeec4ff75bef7b5737fdf4b0c007123d5d5b790b9d0ebfa4028645634870"}
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.614453    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" event={"ID":"4fea3fb8-425c-4000-8529-79f3f4ece0b0","Type":"ContainerDied","Data":"495acdffc99b5dbf3bf1bb8cc321d543f0429aa8968afc127ee0ff39cb91c24c"}
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.614479    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" event={"ID":"a5951710-8983-438c-ae0a-a7749eb1cd32","Type":"ContainerDied","Data":"e0b0d91945c504e9f104cbfe4b0e4d916846ad6dc4244e0d56c721a9dd0b478a"}
Nov 02 22:20:23.616010 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.614523    5580 scope.go:122] "RemoveContainer" containerID="b1a0f5515b9421d48add551c103edea95cb248f6ae25dc961542b97073852340"
Nov 02 22:20:23.670436 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.667971    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 22:20:23.744405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.744373    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:20:23.771832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.771793    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.771866    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.771916    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.771951    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772065 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772041    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772110    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772160    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772240    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772317 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772303    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772335    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772374    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772411    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772477    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772538    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772583    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\") pod \"74d46a2f-20e1-4b5b-b169-e8610095770f\" (UID: \"74d46a2f-20e1-4b5b-b169-e8610095770f\") "
Nov 02 22:20:23.772845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772618    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772659    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772698    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\") pod \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\" (UID: \"1d111d11-ff27-423b-b38c-2e522b0ae4d3\") "
Nov 02 22:20:23.772845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.772801    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:23.773513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773357    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74d46a2f-20e1-4b5b-b169-e8610095770f-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.773513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773358    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.773513 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773377    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.773714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773535    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.773714 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773696    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.773827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773724    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.776998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773940    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.776998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.774211    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:23.776998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.773909    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.776998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.774611    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.778220    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.778782    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.778827    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.778999    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.779031    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.780170    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq" pod "74d46a2f-20e1-4b5b-b169-e8610095770f" (UID: "74d46a2f-20e1-4b5b-b169-e8610095770f"). InnerVolumeSpecName "kube-api-access-tzfmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.781060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.780482    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "kube-api-access-4zlzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.781503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.781110    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy" pod "1d111d11-ff27-423b-b38c-2e522b0ae4d3" (UID: "1d111d11-ff27-423b-b38c-2e522b0ae4d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.796812 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.796774    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:20:23.856442 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.856140    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:20:23.864866 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.864832    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:20:23.873759 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.873729    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.873990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.873770    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.873990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.873843    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.873990 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.873871    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.874204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874115    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.874300 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874272    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:23.874367 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874277    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.874529 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874472    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.874529 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874395    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874575    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874625    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874737    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874745    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\") pod \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\" (UID: \"4fea3fb8-425c-4000-8529-79f3f4ece0b0\") "
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.874855    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875218    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875241    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875257    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875274    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875288    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875302    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875317    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zlzt\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-kube-api-access-4zlzt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875336    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875349    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875364    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875379    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.875383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875392    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875405    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzfmq\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-kube-api-access-tzfmq\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875419    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875432    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875446    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d111d11-ff27-423b-b38c-2e522b0ae4d3-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875460    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875473    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875486    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875500    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875513    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d111d11-ff27-423b-b38c-2e522b0ae4d3-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.876303 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.875528    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74d46a2f-20e1-4b5b-b169-e8610095770f-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.878355 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.878328    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.878467 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.878336    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:23.878608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.878589    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.879674 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.879648    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8" pod "4fea3fb8-425c-4000-8529-79f3f4ece0b0" (UID: "4fea3fb8-425c-4000-8529-79f3f4ece0b0"). InnerVolumeSpecName "kube-api-access-q6xw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.976202 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976170    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.976383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976250    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976301    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976344    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976379    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976409    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976441    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976468    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976495    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976514    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976527    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.976609 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976594    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976672    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976717    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976730    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976736    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976763    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976806    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976876    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976889    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976922    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\") pod \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\" (UID: \"c790de74-3dd8-4fb4-b60b-91493b9b0dbc\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.976982    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.977011 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977014    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977040    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\") pod \"a5951710-8983-438c-ae0a-a7749eb1cd32\" (UID: \"a5951710-8983-438c-ae0a-a7749eb1cd32\") "
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977286    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977592    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977616    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977632    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977646    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977660    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977664    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977674    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977711    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977728    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6xw8\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-kube-api-access-q6xw8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977744    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977763    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.977778    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fea3fb8-425c-4000-8529-79f3f4ece0b0-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:23.978090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.978057    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.978988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.978237    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.979415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.979383    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:23.979547 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.979480    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:23.979647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.979585    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "kube-api-access-r6l7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.979749 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.979724    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:23.979941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.979918    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token" pod "c790de74-3dd8-4fb4-b60b-91493b9b0dbc" (UID: "c790de74-3dd8-4fb4-b60b-91493b9b0dbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.980529 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.980510    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.980835 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.980820    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "kube-api-access-74nnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:23.981374 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:23.981351    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy" pod "a5951710-8983-438c-ae0a-a7749eb1cd32" (UID: "a5951710-8983-438c-ae0a-a7749eb1cd32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:24.078501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078462    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078494    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6l7g\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-kube-api-access-r6l7g\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078505    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5951710-8983-438c-ae0a-a7749eb1cd32-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078519    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078532    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74nnj\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-kube-api-access-74nnj\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078541    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078549    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078558    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c790de74-3dd8-4fb4-b60b-91493b9b0dbc-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078567    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078575    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.078865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.078584    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5951710-8983-438c-ae0a-a7749eb1cd32-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:24.617507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.617418    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"
Nov 02 22:20:24.617507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.617452    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4" event={"ID":"a5951710-8983-438c-ae0a-a7749eb1cd32","Type":"ContainerDied","Data":"40ed742bc50ea2cdbc67aeed795680280120d632672651518f24e51091ce7975"}
Nov 02 22:20:24.618057 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.617516    5580 scope.go:122] "RemoveContainer" containerID="e0b0d91945c504e9f104cbfe4b0e4d916846ad6dc4244e0d56c721a9dd0b478a"
Nov 02 22:20:24.619168 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.619147    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"
Nov 02 22:20:24.619301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.619169    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng" event={"ID":"1d111d11-ff27-423b-b38c-2e522b0ae4d3","Type":"ContainerDied","Data":"ca069f39f107e66c463b2a9b854c5762ffc31f4427a1a1255d55002c0d3f5986"}
Nov 02 22:20:24.620742 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.620719    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"
Nov 02 22:20:24.620836 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.620726    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86" event={"ID":"74d46a2f-20e1-4b5b-b169-e8610095770f","Type":"ContainerDied","Data":"56db105449ef1912a748ca41286770dbe618f81111e6fe273b1af18499e8ca96"}
Nov 02 22:20:24.622208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.622190    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"
Nov 02 22:20:24.622311 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.622233    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d" event={"ID":"4fea3fb8-425c-4000-8529-79f3f4ece0b0","Type":"ContainerDied","Data":"0cb1e4491ebbbe12bb0ae8c1e4cfa74974ec837a4453784765819dbbb0d51c93"}
Nov 02 22:20:24.632974 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.632923    5580 generic.go:356] "Generic (PLEG): container finished" podID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerID="5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d" exitCode=0
Nov 02 22:20:24.633167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.633008    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" event={"ID":"c790de74-3dd8-4fb4-b60b-91493b9b0dbc","Type":"ContainerDied","Data":"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"}
Nov 02 22:20:24.633167 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.633108    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9" event={"ID":"c790de74-3dd8-4fb4-b60b-91493b9b0dbc","Type":"ContainerDied","Data":"8492a4ffeb9eaabe0a5d385a37b59aa9c181534e9cfea688d47f1bef16d1ad5a"}
Nov 02 22:20:24.633280 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.633227    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"
Nov 02 22:20:24.653277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.641578    5580 scope.go:122] "RemoveContainer" containerID="2a7214109965d944754327e2089097f5768cf80d539a7d8d13b8914c387ebd12"
Nov 02 22:20:24.674425 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.674403    5580 scope.go:122] "RemoveContainer" containerID="ac34eeec4ff75bef7b5737fdf4b0c007123d5d5b790b9d0ebfa4028645634870"
Nov 02 22:20:24.684398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.684369    5580 scope.go:122] "RemoveContainer" containerID="495acdffc99b5dbf3bf1bb8cc321d543f0429aa8968afc127ee0ff39cb91c24c"
Nov 02 22:20:24.694260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.694239    5580 scope.go:122] "RemoveContainer" containerID="5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"
Nov 02 22:20:24.705779 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.705579    5580 scope.go:122] "RemoveContainer" containerID="5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"
Nov 02 22:20:24.706293 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:24.706264    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d\": container with ID starting with 5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d not found: ID does not exist" containerID="5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"
Nov 02 22:20:24.706403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.706308    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d"} err="failed to get container status \"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d\": rpc error: code = NotFound desc = could not find container \"5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d\": container with ID starting with 5d78547eb77fa821df24b58ce9dffcd249fbc260462de62767648ee98fa9882d not found: ID does not exist"
Nov 02 22:20:24.833885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.833844    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 22:20:24.883357 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.883274    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9cc56b-openshift-default-6d75946c77-nxlng"]
Nov 02 22:20:24.947073 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.947037    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:20:24.948902 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:24.948878    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-87q86"]
Nov 02 22:20:25.057864 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.057817    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:20:25.068380 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.068338    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-k6fj4"]
Nov 02 22:20:25.183450 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.183411    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:20:25.186642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.186613    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-24z5d"]
Nov 02 22:20:25.296503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.296465    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:20:25.305259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.305229    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-kq8f9"]
Nov 02 22:20:25.434439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.434346    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:20:25.593443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.593401    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" path="/var/lib/kubelet/pods/1d111d11-ff27-423b-b38c-2e522b0ae4d3/volumes"
Nov 02 22:20:25.593933 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.593917    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" path="/var/lib/kubelet/pods/4fea3fb8-425c-4000-8529-79f3f4ece0b0/volumes"
Nov 02 22:20:25.594386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.594372    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" path="/var/lib/kubelet/pods/74d46a2f-20e1-4b5b-b169-e8610095770f/volumes"
Nov 02 22:20:25.595074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.595054    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" path="/var/lib/kubelet/pods/a5951710-8983-438c-ae0a-a7749eb1cd32/volumes"
Nov 02 22:20:25.595484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:25.595470    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" path="/var/lib/kubelet/pods/c790de74-3dd8-4fb4-b60b-91493b9b0dbc/volumes"
Nov 02 22:20:26.312480 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:26.312444    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:20:27.657478 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.657422    5580 generic.go:356] "Generic (PLEG): container finished" podID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerID="bfcce6809f2fff02cc8f2b28a9390d8fe42457f61b28e46d0796359ca675acf9" exitCode=0
Nov 02 22:20:27.658015 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.657483    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" event={"ID":"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb","Type":"ContainerDied","Data":"bfcce6809f2fff02cc8f2b28a9390d8fe42457f61b28e46d0796359ca675acf9"}
Nov 02 22:20:27.804606 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.804525    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:20:27.923048 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.923006    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\") pod \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") "
Nov 02 22:20:27.923260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.923075    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\" (UniqueName: \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\") pod \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\" (UID: \"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb\") "
Nov 02 22:20:27.923445 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.923419    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume" pod "e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" (UID: "e975a0b5-a29e-4eac-bcfc-0e5530f85cdb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:27.923635 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.923621    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:27.925430 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:27.925401    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c" pod "e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" (UID: "e975a0b5-a29e-4eac-bcfc-0e5530f85cdb"). InnerVolumeSpecName "kube-api-access-rjf2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:28.024488 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.024442    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjf2c\" (UniqueName: \"kubernetes.io/projected/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb-kube-api-access-rjf2c\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:28.664537 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.664503    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"
Nov 02 22:20:28.664537 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.664517    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc" event={"ID":"e975a0b5-a29e-4eac-bcfc-0e5530f85cdb","Type":"ContainerDied","Data":"67ed4e6d35963ffe5eda0910ed477c72bc157d277a189bb98833aafb4bb965c2"}
Nov 02 22:20:28.665063 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.664587    5580 scope.go:122] "RemoveContainer" containerID="bfcce6809f2fff02cc8f2b28a9390d8fe42457f61b28e46d0796359ca675acf9"
Nov 02 22:20:28.832662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.832619    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:20:28.881114 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:28.881079    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-af9e7c-79875b6d79-g75gc"]
Nov 02 22:20:29.594129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:29.594095    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" path="/var/lib/kubelet/pods/e975a0b5-a29e-4eac-bcfc-0e5530f85cdb/volumes"
Nov 02 22:20:31.753076 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:31.753035    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:20:32.176965 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:32.176924    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:20:32.177215 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:32.177174    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" containerID="cri-o://dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3" gracePeriod=30
Nov 02 22:20:36.412510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:36.412462    5580 patch_prober.go:29] interesting pod/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:20:36.412926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:36.412529    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:20:37.441859 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.441791    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:20:37.534217 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534180    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534232    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534290    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534376    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534432    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534482    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534578    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534608    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534686 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534666    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:37.534869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534686    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:20:37.534869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534742    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\") pod \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\" (UID: \"b8bf8038-6dfd-4c72-85ac-1a222b061fa4\") "
Nov 02 22:20:37.534869 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534844    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:37.535081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.534993    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:37.535081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535010    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:37.535274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535249    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.535274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535276    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.535480 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535290    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.535480 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535301    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.535480 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.535312    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.536885 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.536863    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:37.536997 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.536898    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "kube-api-access-9hhbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:37.537100 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.537078    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:20:37.537208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.537191    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token" pod "b8bf8038-6dfd-4c72-85ac-1a222b061fa4" (UID: "b8bf8038-6dfd-4c72-85ac-1a222b061fa4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:37.636423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.636386    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.636423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.636420    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hhbp\" (UniqueName: \"kubernetes.io/projected/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-kube-api-access-9hhbp\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.636423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.636430    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.636691 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.636438    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8bf8038-6dfd-4c72-85ac-1a222b061fa4-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:37.716821 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.716733    5580 generic.go:356] "Generic (PLEG): container finished" podID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerID="dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3" exitCode=0
Nov 02 22:20:37.716821 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.716791    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"
Nov 02 22:20:37.717038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.716816    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerDied","Data":"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"}
Nov 02 22:20:37.717038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.716872    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg" event={"ID":"b8bf8038-6dfd-4c72-85ac-1a222b061fa4","Type":"ContainerDied","Data":"283fa8bf73a020c690af1341044ba95f03570334cca9012911050677287175d8"}
Nov 02 22:20:37.717038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.716894    5580 scope.go:122] "RemoveContainer" containerID="dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"
Nov 02 22:20:37.726624 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.726602    5580 scope.go:122] "RemoveContainer" containerID="d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"
Nov 02 22:20:37.736043 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.736017    5580 scope.go:122] "RemoveContainer" containerID="dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"
Nov 02 22:20:37.736402 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:37.736381    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3\": container with ID starting with dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3 not found: ID does not exist" containerID="dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"
Nov 02 22:20:37.736446 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.736412    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3"} err="failed to get container status \"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3\": rpc error: code = NotFound desc = could not find container \"dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3\": container with ID starting with dc391595f6c9e0e191e1ad62aad5fe92b7c23a4a42ba5318bf782fca9cdaaee3 not found: ID does not exist"
Nov 02 22:20:37.736446 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.736434    5580 scope.go:122] "RemoveContainer" containerID="d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"
Nov 02 22:20:37.736709 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:37.736680    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d\": container with ID starting with d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d not found: ID does not exist" containerID="d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"
Nov 02 22:20:37.736826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.736715    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d"} err="failed to get container status \"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d\": rpc error: code = NotFound desc = could not find container \"d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d\": container with ID starting with d0200f9ff6b1a88b9c1dad7accd52c0e59a20214efd45e2d4a08f1ec3bed522d not found: ID does not exist"
Nov 02 22:20:37.889129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.889091    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:20:37.917864 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:37.917831    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-af9e7c-openshift-default-54b6ff44ff-rwxwg"]
Nov 02 22:20:39.593366 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:39.593332    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" path="/var/lib/kubelet/pods/b8bf8038-6dfd-4c72-85ac-1a222b061fa4/volumes"
Nov 02 22:20:43.347034 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:43.346987    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:20:50.980285 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:50.980245    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:20:56.312180 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:56.312134    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:20:58.368141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.368081    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerName="main" containerID="cri-o://f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc" gracePeriod=45
Nov 02 22:20:58.623047 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.622979    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:20:58.641121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641088    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641172    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641198    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\" (UniqueName: \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641236    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641306 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641292    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641317    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\" (UniqueName: \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\") pod \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\" (UID: \"a537edb1-4a8e-42d7-bd82-3ca736aa1969\") "
Nov 02 22:20:58.641532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641376    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:58.641532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641488    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:58.641699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641605    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:58.641775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641737    5580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tmp-dir\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.641775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641770    5580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-model-cache\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.641775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.641781    5580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-home\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.643629 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.643600    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:20:58.643773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.643692    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:20:58.643773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.643691    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs" pod "a537edb1-4a8e-42d7-bd82-3ca736aa1969" (UID: "a537edb1-4a8e-42d7-bd82-3ca736aa1969"). InnerVolumeSpecName "kube-api-access-h5hfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:20:58.743645 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.743606    5580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a537edb1-4a8e-42d7-bd82-3ca736aa1969-tls-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.743645 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.743638    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5hfs\" (UniqueName: \"kubernetes.io/projected/a537edb1-4a8e-42d7-bd82-3ca736aa1969-kube-api-access-h5hfs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.743645 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.743650    5580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a537edb1-4a8e-42d7-bd82-3ca736aa1969-dshm\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:20:58.834575 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.834540    5580 generic.go:356] "Generic (PLEG): container finished" podID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerID="f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc" exitCode=0
Nov 02 22:20:58.834770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.834608    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"
Nov 02 22:20:58.834770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.834639    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" event={"ID":"a537edb1-4a8e-42d7-bd82-3ca736aa1969","Type":"ContainerDied","Data":"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"}
Nov 02 22:20:58.834770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.834688    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" event={"ID":"a537edb1-4a8e-42d7-bd82-3ca736aa1969","Type":"ContainerDied","Data":"06989eac345aa1f6f0f511737526796175de30bc2c285de90c67cd546030f463"}
Nov 02 22:20:58.834770 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.834710    5580 scope.go:122] "RemoveContainer" containerID="f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"
Nov 02 22:20:58.845411 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.845386    5580 scope.go:122] "RemoveContainer" containerID="f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"
Nov 02 22:20:58.845745 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:20:58.845711    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc\": container with ID starting with f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc not found: ID does not exist" containerID="f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"
Nov 02 22:20:58.845853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.845752    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc"} err="failed to get container status \"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc\": rpc error: code = NotFound desc = could not find container \"f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc\": container with ID starting with f916c8c844e45dd8255fcd790a4af3c56f59aae362a5a1544b80fd36664c71cc not found: ID does not exist"
Nov 02 22:20:58.996521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.996479    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:20:58.999470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:58.999437    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp"]
Nov 02 22:20:59.556172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:59.556113    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-d1a155/test-model-d1a155-kserve-58f8688fd5-flgbp" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.144:8000/ready\": context deadline exceeded"
Nov 02 22:20:59.593807 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:20:59.593764    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" path="/var/lib/kubelet/pods/a537edb1-4a8e-42d7-bd82-3ca736aa1969/volumes"
Nov 02 22:21:14.502829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:14.502789    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:21:26.311886 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:26.311848    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:21:28.000237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:28.000194    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:21:33.000578 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.000497    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" podUID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" containerName="payload-processing" containerID="cri-o://56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9" gracePeriod=25
Nov 02 22:21:33.278154 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.278125    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:21:33.304760 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.304720    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\") pod \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") "
Nov 02 22:21:33.304951 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.304851    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\" (UniqueName: \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\") pod \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\" (UID: \"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf\") "
Nov 02 22:21:33.305164 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.305140    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume" pod "8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" (UID: "8b79f34c-f2bb-4857-81dc-a4e025f2cdbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:21:33.307092 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.307058    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss" pod "8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" (UID: "8b79f34c-f2bb-4857-81dc-a4e025f2cdbf"). InnerVolumeSpecName "kube-api-access-d9hss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:33.406208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.406167    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:33.406208 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.406204    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9hss\" (UniqueName: \"kubernetes.io/projected/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf-kube-api-access-d9hss\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:33.442754 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.442715    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:21:33.460330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:33.460277    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:21:34.026659 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.026630    5580 generic.go:356] "Generic (PLEG): container finished" podID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" containerID="56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9" exitCode=0
Nov 02 22:21:34.027110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.026676    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"
Nov 02 22:21:34.027110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.026717    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" event={"ID":"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf","Type":"ContainerDied","Data":"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"}
Nov 02 22:21:34.027110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.026776    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26" event={"ID":"8b79f34c-f2bb-4857-81dc-a4e025f2cdbf","Type":"ContainerDied","Data":"263a543b02d749697a034a79e90b024b3599fcb42a77eb8343a57590c3a22730"}
Nov 02 22:21:34.027110 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.026805    5580 scope.go:122] "RemoveContainer" containerID="56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"
Nov 02 22:21:34.037281 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.037251    5580 scope.go:122] "RemoveContainer" containerID="56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"
Nov 02 22:21:34.037699 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:34.037680    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9\": container with ID starting with 56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9 not found: ID does not exist" containerID="56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"
Nov 02 22:21:34.037739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.037714    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9"} err="failed to get container status \"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9\": rpc error: code = NotFound desc = could not find container \"56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9\": container with ID starting with 56aa080d0cc2c6712e3c0bce82d23991de44e1407300242feb6129b0cbb580f9 not found: ID does not exist"
Nov 02 22:21:34.189483 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.189447    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:21:34.190882 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.190860    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-267d89-846d86c75-8bv26"]
Nov 02 22:21:34.342833 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:34.342743    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:21:35.593988 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:35.593940    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" path="/var/lib/kubelet/pods/8b79f34c-f2bb-4857-81dc-a4e025f2cdbf/volumes"
Nov 02 22:21:38.443518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.443466    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" podUID="c66a6c3f-1624-47d2-876b-96da0b6972a0" containerName="payload-processing" containerID="cri-o://804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b" gracePeriod=25
Nov 02 22:21:38.461046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.460991    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing" containerID="cri-o://c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0" gracePeriod=25
Nov 02 22:21:38.707965 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.707880    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:21:38.763932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.763896    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\" (UniqueName: \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\") pod \"c66a6c3f-1624-47d2-876b-96da0b6972a0\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") "
Nov 02 22:21:38.764183 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.764056    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\") pod \"c66a6c3f-1624-47d2-876b-96da0b6972a0\" (UID: \"c66a6c3f-1624-47d2-876b-96da0b6972a0\") "
Nov 02 22:21:38.764442 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.764413    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume" pod "c66a6c3f-1624-47d2-876b-96da0b6972a0" (UID: "c66a6c3f-1624-47d2-876b-96da0b6972a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:21:38.766199 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.766172    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q" pod "c66a6c3f-1624-47d2-876b-96da0b6972a0" (UID: "c66a6c3f-1624-47d2-876b-96da0b6972a0"). InnerVolumeSpecName "kube-api-access-xxx4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:38.865444 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.865306    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxx4q\" (UniqueName: \"kubernetes.io/projected/c66a6c3f-1624-47d2-876b-96da0b6972a0-kube-api-access-xxx4q\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:38.865444 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:38.865441    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66a6c3f-1624-47d2-876b-96da0b6972a0-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:39.054693 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.054594    5580 generic.go:356] "Generic (PLEG): container finished" podID="c66a6c3f-1624-47d2-876b-96da0b6972a0" containerID="804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b" exitCode=0
Nov 02 22:21:39.054693 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.054643    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"
Nov 02 22:21:39.054693 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.054680    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" event={"ID":"c66a6c3f-1624-47d2-876b-96da0b6972a0","Type":"ContainerDied","Data":"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"}
Nov 02 22:21:39.054994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.054737    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p" event={"ID":"c66a6c3f-1624-47d2-876b-96da0b6972a0","Type":"ContainerDied","Data":"bfd4baec301891a5c13ce0c4674d090beceabc110e2833aa444f7d517b03c924"}
Nov 02 22:21:39.054994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.054764    5580 scope.go:122] "RemoveContainer" containerID="804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"
Nov 02 22:21:39.064085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.064067    5580 scope.go:122] "RemoveContainer" containerID="804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"
Nov 02 22:21:39.064375 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:39.064350    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b\": container with ID starting with 804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b not found: ID does not exist" containerID="804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"
Nov 02 22:21:39.064417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.064386    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b"} err="failed to get container status \"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b\": rpc error: code = NotFound desc = could not find container \"804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b\": container with ID starting with 804fa2b88f23ec91c9dfd7d07b397cf63dc6cb1513de2246420b92047a6e266b not found: ID does not exist"
Nov 02 22:21:39.220656 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.218834    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:21:39.266764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.266722    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d1a155-7b6d6f8959-8n52p"]
Nov 02 22:21:39.592684 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:39.592647    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c66a6c3f-1624-47d2-876b-96da0b6972a0" path="/var/lib/kubelet/pods/c66a6c3f-1624-47d2-876b-96da0b6972a0/volumes"
Nov 02 22:21:44.135403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:44.135364    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:21:44.135921 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:44.135648    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy" containerID="cri-o://55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab" gracePeriod=30
Nov 02 22:21:44.505923 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:44.505880    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:21:49.393640 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.393609    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:21:49.476597 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476556    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476606    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476637    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476669    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476703    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476741    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.476796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476790    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.477147 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476836    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.477147 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476869    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\") pod \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\" (UID: \"a3a2beaf-9cdc-417e-9154-ccb65c5a375a\") "
Nov 02 22:21:49.477147 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.476914    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:49.477147 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477091    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:49.477352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477291    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:49.477415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477401    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:49.477474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477336    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:21:49.477612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477561    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.477612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477590    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.477612 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477604    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.477764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477617    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.477764 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.477633    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.479235 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.479206    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:21:49.479370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.479253    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "kube-api-access-rfg7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:49.479510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.479485    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:49.479603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.479522    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy" pod "a3a2beaf-9cdc-417e-9154-ccb65c5a375a" (UID: "a3a2beaf-9cdc-417e-9154-ccb65c5a375a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:49.579150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.579112    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.579150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.579150    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.579370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.579162    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfg7s\" (UniqueName: \"kubernetes.io/projected/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-kube-api-access-rfg7s\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:49.579370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:49.579173    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3a2beaf-9cdc-417e-9154-ccb65c5a375a-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:50.118872 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.118835    5580 generic.go:356] "Generic (PLEG): container finished" podID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerID="55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab" exitCode=0
Nov 02 22:21:50.119075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.118881    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"
Nov 02 22:21:50.119075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.118914    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerDied","Data":"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"}
Nov 02 22:21:50.119075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.118998    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp" event={"ID":"a3a2beaf-9cdc-417e-9154-ccb65c5a375a","Type":"ContainerDied","Data":"1922531db0aa510acef61b2540114d72b7a484b5f9eca317278094bb5a93ba34"}
Nov 02 22:21:50.119075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.119029    5580 scope.go:122] "RemoveContainer" containerID="55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"
Nov 02 22:21:50.128205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.128183    5580 scope.go:122] "RemoveContainer" containerID="50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"
Nov 02 22:21:50.137175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.137151    5580 scope.go:122] "RemoveContainer" containerID="55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"
Nov 02 22:21:50.137515 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:50.137492    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab\": container with ID starting with 55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab not found: ID does not exist" containerID="55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"
Nov 02 22:21:50.137558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.137529    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab"} err="failed to get container status \"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab\": rpc error: code = NotFound desc = could not find container \"55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab\": container with ID starting with 55adab1e22f460613557702daa210eae9ec4cc576d9d29112c16c5759d8fb3ab not found: ID does not exist"
Nov 02 22:21:50.137558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.137549    5580 scope.go:122] "RemoveContainer" containerID="50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"
Nov 02 22:21:50.137800 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:50.137787    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936\": container with ID starting with 50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936 not found: ID does not exist" containerID="50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"
Nov 02 22:21:50.137843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.137803    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936"} err="failed to get container status \"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936\": rpc error: code = NotFound desc = could not find container \"50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936\": container with ID starting with 50b393a8c3f484325c8baf1d2475ff5bbc289e1435981d98410d342c4492b936 not found: ID does not exist"
Nov 02 22:21:50.330648 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.330613    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:21:50.379747 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:50.379660    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d1a155-openshift-default-5c7c77df85-9j6wp"]
Nov 02 22:21:51.593043 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:51.593009    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" path="/var/lib/kubelet/pods/a3a2beaf-9cdc-417e-9154-ccb65c5a375a/volumes"
Nov 02 22:21:52.686249 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:52.686214    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:21:52.686703 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:52.686421    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" containerID="cri-o://45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa" gracePeriod=30
Nov 02 22:21:53.304132 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:53.304090    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:21:54.506608 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:54.506571    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:21:56.302781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:56.302741    5580 patch_prober.go:29] interesting pod/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:21:56.303211 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:56.302809    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:21:56.313166 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:56.313134    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:21:57.957435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:57.957372    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:21:58.072832 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.072793    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.072889    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.072913    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.072938    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073032    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073081 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073063    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073378 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073098    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073378 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073238    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073378 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073274    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:58.073378 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073297    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\") pod \"3762c23d-3be1-47c7-bfda-024fd935d42e\" (UID: \"3762c23d-3be1-47c7-bfda-024fd935d42e\") "
Nov 02 22:21:58.073378 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073304    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:58.073651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073404    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:58.073651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073445    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:58.073748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.073643    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.074238    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.074279    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.074296    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3762c23d-3be1-47c7-bfda-024fd935d42e-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.074312    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.074326    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.077104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.076112    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:21:58.083917 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.083880    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:21:58.084224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.084194    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "kube-api-access-b7ksl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:58.084396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.084377    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token" pod "3762c23d-3be1-47c7-bfda-024fd935d42e" (UID: "3762c23d-3be1-47c7-bfda-024fd935d42e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:58.170767 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.170734    5580 generic.go:356] "Generic (PLEG): container finished" podID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerID="45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa" exitCode=0
Nov 02 22:21:58.170970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.170814    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"
Nov 02 22:21:58.170970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.170814    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerDied","Data":"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"}
Nov 02 22:21:58.170970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.170907    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj" event={"ID":"3762c23d-3be1-47c7-bfda-024fd935d42e","Type":"ContainerDied","Data":"5e3a5ad10d3653e1bcdf72e08fc9840191a270f0edcd661df45c5204efd657db"}
Nov 02 22:21:58.170970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.170939    5580 scope.go:122] "RemoveContainer" containerID="45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"
Nov 02 22:21:58.175148 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.175124    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7ksl\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-kube-api-access-b7ksl\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.175258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.175151    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.175258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.175168    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.175258 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.175183    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3762c23d-3be1-47c7-bfda-024fd935d42e-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.180269 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.180245    5580 scope.go:122] "RemoveContainer" containerID="caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"
Nov 02 22:21:58.189519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.189498    5580 scope.go:122] "RemoveContainer" containerID="45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"
Nov 02 22:21:58.189857 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:58.189833    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa\": container with ID starting with 45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa not found: ID does not exist" containerID="45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"
Nov 02 22:21:58.189907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.189870    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa"} err="failed to get container status \"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa\": rpc error: code = NotFound desc = could not find container \"45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa\": container with ID starting with 45ef6a4b99102cceec59ec2ab89ec419a9cdcaec8c4438b30b8f0b83b0263aaa not found: ID does not exist"
Nov 02 22:21:58.189907 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.189898    5580 scope.go:122] "RemoveContainer" containerID="caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"
Nov 02 22:21:58.190307 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:58.190261    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda\": container with ID starting with caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda not found: ID does not exist" containerID="caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"
Nov 02 22:21:58.190307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.190295    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda"} err="failed to get container status \"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda\": rpc error: code = NotFound desc = could not find container \"caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda\": container with ID starting with caa2d25b46f9b5288a3af80447290923b4ddbd23bbd93653505ce4e759abbfda not found: ID does not exist"
Nov 02 22:21:58.334235 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.334145    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:21:58.338184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.338151    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-267d89-openshift-default-d8779f489-gffqj"]
Nov 02 22:21:58.701803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.701775    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:21:58.781162 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.781124    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\" (UniqueName: \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\") pod \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") "
Nov 02 22:21:58.781385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.781222    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\") pod \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\" (UID: \"e8a71af5-1568-4c78-a7cf-70d0eb7defe1\") "
Nov 02 22:21:58.781631 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.781606    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume" pod "e8a71af5-1568-4c78-a7cf-70d0eb7defe1" (UID: "e8a71af5-1568-4c78-a7cf-70d0eb7defe1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:21:58.783440 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.783422    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8" pod "e8a71af5-1568-4c78-a7cf-70d0eb7defe1" (UID: "e8a71af5-1568-4c78-a7cf-70d0eb7defe1"). InnerVolumeSpecName "kube-api-access-qh9h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:21:58.882307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.882268    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh9h8\" (UniqueName: \"kubernetes.io/projected/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-kube-api-access-qh9h8\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:58.882307 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:58.882305    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a71af5-1568-4c78-a7cf-70d0eb7defe1-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:21:59.178216 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.178172    5580 generic.go:356] "Generic (PLEG): container finished" podID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerID="c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0" exitCode=0
Nov 02 22:21:59.178216 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.178217    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"
Nov 02 22:21:59.178803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.178256    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" event={"ID":"e8a71af5-1568-4c78-a7cf-70d0eb7defe1","Type":"ContainerDied","Data":"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"}
Nov 02 22:21:59.178803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.178316    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4" event={"ID":"e8a71af5-1568-4c78-a7cf-70d0eb7defe1","Type":"ContainerDied","Data":"f1c223c40dee3609edecbb7069422b3c2ad6857c1f5f44964ef22081af3e3f00"}
Nov 02 22:21:59.178803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.178342    5580 scope.go:122] "RemoveContainer" containerID="c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"
Nov 02 22:21:59.187941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.187920    5580 scope.go:122] "RemoveContainer" containerID="c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"
Nov 02 22:21:59.188341 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:21:59.188317    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0\": container with ID starting with c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0 not found: ID does not exist" containerID="c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"
Nov 02 22:21:59.188391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.188354    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0"} err="failed to get container status \"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0\": rpc error: code = NotFound desc = could not find container \"c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0\": container with ID starting with c549fc192f9d6bc57db6c9b1a4f7ac0b41466db157b4f15dbceb0332514ff6c0 not found: ID does not exist"
Nov 02 22:21:59.339599 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.339555    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:21:59.389250 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.389211    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d1a155-6995556487-2pmp4"]
Nov 02 22:21:59.592657 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.592561    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" path="/var/lib/kubelet/pods/3762c23d-3be1-47c7-bfda-024fd935d42e/volumes"
Nov 02 22:21:59.593020 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:21:59.593005    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" path="/var/lib/kubelet/pods/e8a71af5-1568-4c78-a7cf-70d0eb7defe1/volumes"
Nov 02 22:22:00.272295 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.272256    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:22:00.272717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.272482    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" podUID="0c11894c-5c60-43e9-971e-8533012685c2" containerName="maas-api" containerID="cri-o://eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289" gracePeriod=30
Nov 02 22:22:00.542048 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.542018    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:22:00.716655 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.716610    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\") pod \"0c11894c-5c60-43e9-971e-8533012685c2\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") "
Nov 02 22:22:00.716655 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.716660    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") pod \"0c11894c-5c60-43e9-971e-8533012685c2\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") "
Nov 02 22:22:00.716940 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.716713    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\") pod \"0c11894c-5c60-43e9-971e-8533012685c2\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") "
Nov 02 22:22:00.716940 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.716770    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\" (UniqueName: \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\") pod \"0c11894c-5c60-43e9-971e-8533012685c2\" (UID: \"0c11894c-5c60-43e9-971e-8533012685c2\") "
Nov 02 22:22:00.717122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.717088    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca" pod "0c11894c-5c60-43e9-971e-8533012685c2" (UID: "0c11894c-5c60-43e9-971e-8533012685c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:22:00.717264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.717246    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c11894c-5c60-43e9-971e-8533012685c2-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:00.719045 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.719015    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs" pod "0c11894c-5c60-43e9-971e-8533012685c2" (UID: "0c11894c-5c60-43e9-971e-8533012685c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:00.719165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.719059    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls" pod "0c11894c-5c60-43e9-971e-8533012685c2" (UID: "0c11894c-5c60-43e9-971e-8533012685c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:00.719165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.719080    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt" pod "0c11894c-5c60-43e9-971e-8533012685c2" (UID: "0c11894c-5c60-43e9-971e-8533012685c2"). InnerVolumeSpecName "kube-api-access-lsbwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:00.818423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.818325    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:00.818423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.818358    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c11894c-5c60-43e9-971e-8533012685c2-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:00.818423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:00.818368    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsbwt\" (UniqueName: \"kubernetes.io/projected/0c11894c-5c60-43e9-971e-8533012685c2-kube-api-access-lsbwt\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:01.192166 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.192130    5580 generic.go:356] "Generic (PLEG): container finished" podID="0c11894c-5c60-43e9-971e-8533012685c2" containerID="eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289" exitCode=0
Nov 02 22:22:01.192383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.192191    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" event={"ID":"0c11894c-5c60-43e9-971e-8533012685c2","Type":"ContainerDied","Data":"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"}
Nov 02 22:22:01.192383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.192199    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"
Nov 02 22:22:01.192383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.192235    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv" event={"ID":"0c11894c-5c60-43e9-971e-8533012685c2","Type":"ContainerDied","Data":"a5b03ec40c18066548265b881c9f6baf090b88e7d3e7fde70a27a89773734a12"}
Nov 02 22:22:01.192383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.192256    5580 scope.go:122] "RemoveContainer" containerID="eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"
Nov 02 22:22:01.202868 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.202849    5580 scope.go:122] "RemoveContainer" containerID="eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"
Nov 02 22:22:01.203224 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:22:01.203199    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289\": container with ID starting with eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289 not found: ID does not exist" containerID="eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"
Nov 02 22:22:01.203333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.203231    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289"} err="failed to get container status \"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289\": rpc error: code = NotFound desc = could not find container \"eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289\": container with ID starting with eba2256acf0fd195e85b8a99f2bf9cbfabdfd8d67297d8f96543de6276bbf289 not found: ID does not exist"
Nov 02 22:22:01.353720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.353679    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:22:01.402811 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.402776    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-94a8a3-85974b977f-6n6rv"]
Nov 02 22:22:01.592748 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:01.592655    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c11894c-5c60-43e9-971e-8533012685c2" path="/var/lib/kubelet/pods/0c11894c-5c60-43e9-971e-8533012685c2/volumes"
Nov 02 22:22:08.346604 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:08.346557    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:22:11.536727 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.536690    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"]
Nov 02 22:22:11.537175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537136    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:11.537175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537148    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:11.537175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537161    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing"
Nov 02 22:22:11.537175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537166    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing"
Nov 02 22:22:11.537175 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537175    5580 cpu_manager.go:414] "containerMap: removing container" podUID="101972f2-0c0f-4f0a-9212-41c61f396fe5" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537180    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="101972f2-0c0f-4f0a-9212-41c61f396fe5" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537190    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537195    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537205    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537210    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537216    5580 cpu_manager.go:414] "containerMap: removing container" podUID="642d3830-840c-45ad-9f91-6553a5f3d03f" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537220    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="642d3830-840c-45ad-9f91-6553a5f3d03f" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537225    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537229    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerName="main"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537239    5580 cpu_manager.go:414] "containerMap: removing container" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537244    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537251    5580 cpu_manager.go:414] "containerMap: removing container" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537256    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537264    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537268    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537275    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c66a6c3f-1624-47d2-876b-96da0b6972a0" containerName="payload-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537279    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c66a6c3f-1624-47d2-876b-96da0b6972a0" containerName="payload-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537286    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537290    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537295    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537300    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537315    5580 cpu_manager.go:414] "containerMap: removing container" podUID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" containerName="maas-api"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537320    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" containerName="maas-api"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537328    5580 cpu_manager.go:414] "containerMap: removing container" podUID="12236a2c-5fca-46cd-aaa7-f05e01670f06" containerName="payload-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537333    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12236a2c-5fca-46cd-aaa7-f05e01670f06" containerName="payload-processing"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537345    5580 cpu_manager.go:414] "containerMap: removing container" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.537343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537350    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537358    5580 cpu_manager.go:414] "containerMap: removing container" podUID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" containerName="payload-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537363    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" containerName="payload-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537369    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537374    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537381    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0c11894c-5c60-43e9-971e-8533012685c2" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537385    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c11894c-5c60-43e9-971e-8533012685c2" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537390    5580 cpu_manager.go:414] "containerMap: removing container" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537394    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537404    5580 cpu_manager.go:414] "containerMap: removing container" podUID="705e02c7-d567-4802-ae53-a1a83f630d9e" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537408    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="705e02c7-d567-4802-ae53-a1a83f630d9e" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537472    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537478    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537485    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5951710-8983-438c-ae0a-a7749eb1cd32" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537494    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b79f34c-f2bb-4857-81dc-a4e025f2cdbf" containerName="payload-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537500    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c11894c-5c60-43e9-971e-8533012685c2" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537504    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="74d46a2f-20e1-4b5b-b169-e8610095770f" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537509    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537514    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537518    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537523    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="705e02c7-d567-4802-ae53-a1a83f630d9e" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537529    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8a71af5-1568-4c78-a7cf-70d0eb7defe1" containerName="payload-pre-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537534    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="101972f2-0c0f-4f0a-9212-41c61f396fe5" containerName="main"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537540    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537544    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537550    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="642d3830-840c-45ad-9f91-6553a5f3d03f" containerName="main"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537555    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a537edb1-4a8e-42d7-bd82-3ca736aa1969" containerName="main"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537560    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d29e7f-e347-453d-8e46-abdc72da06b2" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537565    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fea3fb8-425c-4000-8529-79f3f4ece0b0" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537570    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e975a0b5-a29e-4eac-bcfc-0e5530f85cdb" containerName="payload-pre-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537576    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c66a6c3f-1624-47d2-876b-96da0b6972a0" containerName="payload-processing"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537580    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c790de74-3dd8-4fb4-b60b-91493b9b0dbc" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537584    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537590    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="65528f8c-552e-47dc-af4c-6ac6c8d5324f" containerName="maas-api"
Nov 02 22:22:11.538209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.537597    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="12236a2c-5fca-46cd-aaa7-f05e01670f06" containerName="payload-processing"
Nov 02 22:22:11.542651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.542632    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.597326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.597294    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"]
Nov 02 22:22:11.617639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.617586    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55l9j\" (UniqueName: \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.617851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.617750    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.617851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.617802    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.644406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.644363    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 22:22:11.644597 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.644430    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c4p9p\""
Nov 02 22:22:11.718586 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.718551    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.718775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.718599    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.718775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.718651    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55l9j\" (UniqueName: \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.719301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.719280    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.721035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.721017    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.775170 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.775127    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55l9j\" (UniqueName: \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\") pod \"maas-api-revoke-sub-36900405-g676f\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:11.855785 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:11.855691    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:12.227702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:12.227675    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"]
Nov 02 22:22:12.230548 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:22:12.230515    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e03df3c_4de5_4122_85c0_deb318fc57d3.slice/crio-1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f WatchSource:0}: Error finding container 1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f: Status 404 returned error can't find the container with id 1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f
Nov 02 22:22:12.246510 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:12.246472    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f" event={"ID":"0e03df3c-4de5-4122-85c0-deb318fc57d3","Type":"ContainerStarted","Data":"1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f"}
Nov 02 22:22:13.253664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:13.253630    5580 generic.go:356] "Generic (PLEG): container finished" podID="0e03df3c-4de5-4122-85c0-deb318fc57d3" containerID="8b212ffb4f8cd6725d0915db12ad8a47fc2a6a3a844e3ba005a3190aa9c703a1" exitCode=0
Nov 02 22:22:13.254129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:13.253718    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f" event={"ID":"0e03df3c-4de5-4122-85c0-deb318fc57d3","Type":"ContainerDied","Data":"8b212ffb4f8cd6725d0915db12ad8a47fc2a6a3a844e3ba005a3190aa9c703a1"}
Nov 02 22:22:14.411593 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.411563    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:14.448667 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.448630    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\") pod \"0e03df3c-4de5-4122-85c0-deb318fc57d3\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") "
Nov 02 22:22:14.448875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.448717    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\") pod \"0e03df3c-4de5-4122-85c0-deb318fc57d3\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") "
Nov 02 22:22:14.448875 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.448759    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\" (UniqueName: \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\") pod \"0e03df3c-4de5-4122-85c0-deb318fc57d3\" (UID: \"0e03df3c-4de5-4122-85c0-deb318fc57d3\") "
Nov 02 22:22:14.449160 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.449135    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca" pod "0e03df3c-4de5-4122-85c0-deb318fc57d3" (UID: "0e03df3c-4de5-4122-85c0-deb318fc57d3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:22:14.450817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.450790    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j" pod "0e03df3c-4de5-4122-85c0-deb318fc57d3" (UID: "0e03df3c-4de5-4122-85c0-deb318fc57d3"). InnerVolumeSpecName "kube-api-access-55l9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:14.450903 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.450801    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca" pod "0e03df3c-4de5-4122-85c0-deb318fc57d3" (UID: "0e03df3c-4de5-4122-85c0-deb318fc57d3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:14.550660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.550534    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-xks-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:14.550884 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.550851    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e03df3c-4de5-4122-85c0-deb318fc57d3-maas-api-openshift-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:14.550991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:14.550891    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55l9j\" (UniqueName: \"kubernetes.io/projected/0e03df3c-4de5-4122-85c0-deb318fc57d3-kube-api-access-55l9j\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:15.265223 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:15.265186    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"
Nov 02 22:22:15.265223 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:15.265198    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f" event={"ID":"0e03df3c-4de5-4122-85c0-deb318fc57d3","Type":"ContainerDied","Data":"1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f"}
Nov 02 22:22:15.265454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:15.265283    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d6c36d83972c4370746556f7e60565255ae4531353758bd896416b9c7678e6f"
Nov 02 22:22:26.303773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.303731    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"]
Nov 02 22:22:26.304363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304340    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:26.304520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304502    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d111d11-ff27-423b-b38c-2e522b0ae4d3" containerName="istio-proxy"
Nov 02 22:22:26.304632 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304621    5580 cpu_manager.go:414] "containerMap: removing container" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:26.304709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304698    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8bf8038-6dfd-4c72-85ac-1a222b061fa4" containerName="istio-proxy"
Nov 02 22:22:26.304816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304798    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0e03df3c-4de5-4122-85c0-deb318fc57d3" containerName="revoke-keys"
Nov 02 22:22:26.304816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304811    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e03df3c-4de5-4122-85c0-deb318fc57d3" containerName="revoke-keys"
Nov 02 22:22:26.304816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304822    5580 cpu_manager.go:414] "containerMap: removing container" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:26.305046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304827    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:26.305046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304840    5580 cpu_manager.go:414] "containerMap: removing container" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:26.305046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304845    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3762c23d-3be1-47c7-bfda-024fd935d42e" containerName="istio-proxy"
Nov 02 22:22:26.305046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304966    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3a2beaf-9cdc-417e-9154-ccb65c5a375a" containerName="istio-proxy"
Nov 02 22:22:26.305046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.304978    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e03df3c-4de5-4122-85c0-deb318fc57d3" containerName="revoke-keys"
Nov 02 22:22:26.308213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.308190    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.316571 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:22:26.316541    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:22:26.362532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.362497    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"]
Nov 02 22:22:26.371387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.371349    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.371598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.371500    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.371598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.371548    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx2qg\" (UniqueName: \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.409880 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.409841    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 22:22:26.410078 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.409907    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c4p9p\""
Nov 02 22:22:26.472857 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.472823    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.472857 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.472870    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx2qg\" (UniqueName: \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.473135 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.472907    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.473517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.473493    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.475388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.475364    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.529737 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.529696    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx2qg\" (UniqueName: \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\") pod \"maas-api-revoke-sub-42abe6b5-8q8qt\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.622796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.622697    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:26.999937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:26.999785    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"]
Nov 02 22:22:27.002717 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:22:27.002687    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9d736f_5af8_42bd_9853_ba104020c1fa.slice/crio-b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74 WatchSource:0}: Error finding container b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74: Status 404 returned error can't find the container with id b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74
Nov 02 22:22:27.334318 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:27.334273    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt" event={"ID":"5a9d736f-5af8-42bd-9853-ba104020c1fa","Type":"ContainerStarted","Data":"1277bf14d52cf82c1a28f68c029e79eb877e83d1cfb9b7d4c049b3487397704e"}
Nov 02 22:22:27.334738 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:27.334379    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt" event={"ID":"5a9d736f-5af8-42bd-9853-ba104020c1fa","Type":"ContainerStarted","Data":"b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74"}
Nov 02 22:22:27.450106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:27.450041    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt" podStartSLOduration=1.4500268090000001 podStartE2EDuration="1.450026809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:27.448149151 +0000 UTC m=+2845.964644926" watchObservedRunningTime="2026-11-02 22:22:27.450026809 +0000 UTC m=+2845.966522582"
Nov 02 22:22:28.342016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:28.341944    5580 generic.go:356] "Generic (PLEG): container finished" podID="5a9d736f-5af8-42bd-9853-ba104020c1fa" containerID="1277bf14d52cf82c1a28f68c029e79eb877e83d1cfb9b7d4c049b3487397704e" exitCode=0
Nov 02 22:22:28.342474 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:28.342060    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt" event={"ID":"5a9d736f-5af8-42bd-9853-ba104020c1fa","Type":"ContainerDied","Data":"1277bf14d52cf82c1a28f68c029e79eb877e83d1cfb9b7d4c049b3487397704e"}
Nov 02 22:22:29.484843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.484812    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:29.606053 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.605949    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\" (UniqueName: \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\") pod \"5a9d736f-5af8-42bd-9853-ba104020c1fa\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") "
Nov 02 22:22:29.606053 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.606034    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\") pod \"5a9d736f-5af8-42bd-9853-ba104020c1fa\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") "
Nov 02 22:22:29.606314 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.606137    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\") pod \"5a9d736f-5af8-42bd-9853-ba104020c1fa\" (UID: \"5a9d736f-5af8-42bd-9853-ba104020c1fa\") "
Nov 02 22:22:29.606533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.606507    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca" pod "5a9d736f-5af8-42bd-9853-ba104020c1fa" (UID: "5a9d736f-5af8-42bd-9853-ba104020c1fa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:22:29.608101 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.608075    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca" pod "5a9d736f-5af8-42bd-9853-ba104020c1fa" (UID: "5a9d736f-5af8-42bd-9853-ba104020c1fa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:29.608261 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.608246    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg" pod "5a9d736f-5af8-42bd-9853-ba104020c1fa" (UID: "5a9d736f-5af8-42bd-9853-ba104020c1fa"). InnerVolumeSpecName "kube-api-access-qx2qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:29.707851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.707810    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-openshift-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:29.707851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.707847    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx2qg\" (UniqueName: \"kubernetes.io/projected/5a9d736f-5af8-42bd-9853-ba104020c1fa-kube-api-access-qx2qg\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:29.707851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:29.707859    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a9d736f-5af8-42bd-9853-ba104020c1fa-maas-api-xks-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:30.355254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:30.355217    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt" event={"ID":"5a9d736f-5af8-42bd-9853-ba104020c1fa","Type":"ContainerDied","Data":"b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74"}
Nov 02 22:22:30.355254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:30.355249    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"
Nov 02 22:22:30.355480 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:30.355278    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2fe30c7a5b325d9cd619ee3ae375f2e02dd9590f7301289ef11b718e14dec74"
Nov 02 22:22:32.761678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:32.761640    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:22:40.089482 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:40.089442    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 22:22:40.090033 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:40.089644    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy" containerID="cri-o://8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145" gracePeriod=30
Nov 02 22:22:45.343705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.343674    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 22:22:45.351841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.351815    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.351855    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.351913    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.351938    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352027 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.351993    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352031    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352070    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352098    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352139    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\") pod \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\" (UID: \"9bf2b17e-8cd9-47a3-9d91-40a6f4928634\") "
Nov 02 22:22:45.352264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352152    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:22:45.352533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352332    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:22:45.352533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352347    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:22:45.352660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352547    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:22:45.352660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352577    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.352660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352600    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.352660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352616    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.352842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.352741    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:22:45.354386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.354322    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:45.354386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.354335    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:22:45.354638 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.354621    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:22:45.354691 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.354653    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr" pod "9bf2b17e-8cd9-47a3-9d91-40a6f4928634" (UID: "9bf2b17e-8cd9-47a3-9d91-40a6f4928634"). InnerVolumeSpecName "kube-api-access-b6qbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:45.428750 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.428710    5580 generic.go:356] "Generic (PLEG): container finished" podID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerID="8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145" exitCode=0
Nov 02 22:22:45.428981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.428773    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"
Nov 02 22:22:45.428981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.428802    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" event={"ID":"9bf2b17e-8cd9-47a3-9d91-40a6f4928634","Type":"ContainerDied","Data":"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"}
Nov 02 22:22:45.428981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.428901    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt" event={"ID":"9bf2b17e-8cd9-47a3-9d91-40a6f4928634","Type":"ContainerDied","Data":"f77345146b14deed8738fb2147e6cf1c32e32a7c19aedff91c9bc3d416bc3521"}
Nov 02 22:22:45.428981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.428930    5580 scope.go:122] "RemoveContainer" containerID="8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"
Nov 02 22:22:45.438747 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.438723    5580 scope.go:122] "RemoveContainer" containerID="8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"
Nov 02 22:22:45.439142 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:22:45.439118    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145\": container with ID starting with 8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145 not found: ID does not exist" containerID="8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"
Nov 02 22:22:45.439250 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.439157    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145"} err="failed to get container status \"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145\": rpc error: code = NotFound desc = could not find container \"8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145\": container with ID starting with 8d2a893041558d1546b905591054bac3e7358d5ac2fd3b6e7c024c49557e3145 not found: ID does not exist"
Nov 02 22:22:45.453806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453772    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.453806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453803    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.454003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453816    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6qbr\" (UniqueName: \"kubernetes.io/projected/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-kube-api-access-b6qbr\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.454003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453827    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.454003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453836    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.454003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.453844    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf2b17e-8cd9-47a3-9d91-40a6f4928634-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:45.618284 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.618187    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 22:22:45.621845 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:45.621808    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-10fdd5-openshift-default-ffbc4d897-hn5nt"]
Nov 02 22:22:47.593497 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:47.593460    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" path="/var/lib/kubelet/pods/9bf2b17e-8cd9-47a3-9d91-40a6f4928634/volumes"
Nov 02 22:22:54.217712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.217673    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 22:22:54.218253 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.217975    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager" containerID="cri-o://1a71652067e5238f4e17a4b2f6a397c4f9571d27daf82fb16aa88d3a621fa6e2" gracePeriod=30
Nov 02 22:22:54.475816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.475729    5580 generic.go:356] "Generic (PLEG): container finished" podID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerID="1a71652067e5238f4e17a4b2f6a397c4f9571d27daf82fb16aa88d3a621fa6e2" exitCode=0
Nov 02 22:22:54.475816 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.475781    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerDied","Data":"1a71652067e5238f4e17a4b2f6a397c4f9571d27daf82fb16aa88d3a621fa6e2"}
Nov 02 22:22:54.475999 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.475823    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn" event={"ID":"d4bd6e94-7944-4b09-b6ff-340fbae13b9b","Type":"ContainerDied","Data":"cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8"}
Nov 02 22:22:54.475999 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.475840    5580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbd7c92e37aaf64b10b7d56fc4ce1ab2e2fac5b5ff87df665a038aeabc7acba8"
Nov 02 22:22:54.475999 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.475853    5580 scope.go:122] "RemoveContainer" containerID="e9a0ac87e780a28d3c1de640130351ca97d183e35cbb62a6f8a7d86d378cdb78"
Nov 02 22:22:54.481601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.481573    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 22:22:54.541697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.541661    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\") pod \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") "
Nov 02 22:22:54.541697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.541709    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\") pod \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") "
Nov 02 22:22:54.542002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.541749    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\" (UniqueName: \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\") pod \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\" (UID: \"d4bd6e94-7944-4b09-b6ff-340fbae13b9b\") "
Nov 02 22:22:54.544017 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.543979    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs" pod "d4bd6e94-7944-4b09-b6ff-340fbae13b9b" (UID: "d4bd6e94-7944-4b09-b6ff-340fbae13b9b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:54.544142 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.544033    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert" pod "d4bd6e94-7944-4b09-b6ff-340fbae13b9b" (UID: "d4bd6e94-7944-4b09-b6ff-340fbae13b9b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:22:54.544142 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.544036    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz" pod "d4bd6e94-7944-4b09-b6ff-340fbae13b9b" (UID: "d4bd6e94-7944-4b09-b6ff-340fbae13b9b"). InnerVolumeSpecName "kube-api-access-4qzkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:22:54.643115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.643080    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:54.643115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.643114    5580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-webhook-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:54.643382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:54.643128    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qzkz\" (UniqueName: \"kubernetes.io/projected/d4bd6e94-7944-4b09-b6ff-340fbae13b9b-kube-api-access-4qzkz\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:22:55.482362 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:55.482330    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-jd2nn"
Nov 02 22:22:55.704398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:55.704360    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 22:22:55.708533 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:55.708495    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-jd2nn"]
Nov 02 22:22:56.309872 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:22:56.309833    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:22:56.516649 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.516610    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517003    5580 cpu_manager.go:414] "containerMap: removing container" podUID="5a9d736f-5af8-42bd-9853-ba104020c1fa" containerName="revoke-keys"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517015    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a9d736f-5af8-42bd-9853-ba104020c1fa" containerName="revoke-keys"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517037    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517042    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517061    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517066    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517077    5580 cpu_manager.go:414] "containerMap: removing container" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy"
Nov 02 22:22:56.517107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517082    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy"
Nov 02 22:22:56.517363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517139    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517150    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" containerName="manager"
Nov 02 22:22:56.517363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517158    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bf2b17e-8cd9-47a3-9d91-40a6f4928634" containerName="istio-proxy"
Nov 02 22:22:56.517363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.517165    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a9d736f-5af8-42bd-9853-ba104020c1fa" containerName="revoke-keys"
Nov 02 22:22:56.521653 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.521632    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.561739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.561652    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mxh\" (UniqueName: \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.561739 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.561692    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.561949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.561810    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.573723 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.573693    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 22:22:56.623457 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.623425    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-b76mn\""
Nov 02 22:22:56.623642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.623515    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 22:22:56.623642 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.623561    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 22:22:56.625702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.625669    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:22:56.662870 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.662826    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.663094 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.662907    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mxh\" (UniqueName: \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.663094 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.662933    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.665375 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.665351    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.665535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.665409    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.719787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.719751    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mxh\" (UniqueName: \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\") pod \"maas-controller-5cc9f7769b-pbmv7\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") " pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:56.834550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:56.834453    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:57.211299 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:57.211260    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:22:57.212833 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:22:57.212801    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5eaae0_697a_4cf3_87fd_7f755111e817.slice/crio-00bf4dcd44697c0b185ecc7bf58e10fa425b775f5181b14e6341a8aa7a41a1bb WatchSource:0}: Error finding container 00bf4dcd44697c0b185ecc7bf58e10fa425b775f5181b14e6341a8aa7a41a1bb: Status 404 returned error can't find the container with id 00bf4dcd44697c0b185ecc7bf58e10fa425b775f5181b14e6341a8aa7a41a1bb
Nov 02 22:22:57.213932 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:57.213907    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:22:57.493551 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:57.493457    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" event={"ID":"7e5eaae0-697a-4cf3-87fd-7f755111e817","Type":"ContainerStarted","Data":"00bf4dcd44697c0b185ecc7bf58e10fa425b775f5181b14e6341a8aa7a41a1bb"}
Nov 02 22:22:57.552532 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:57.552495    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:22:57.592545 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:57.592511    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4bd6e94-7944-4b09-b6ff-340fbae13b9b" path="/var/lib/kubelet/pods/d4bd6e94-7944-4b09-b6ff-340fbae13b9b/volumes"
Nov 02 22:22:58.499777 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:58.499746    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" event={"ID":"7e5eaae0-697a-4cf3-87fd-7f755111e817","Type":"ContainerStarted","Data":"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"}
Nov 02 22:22:58.499998 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:58.499854    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:22:58.614933 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:22:58.614875    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" podStartSLOduration=1.719404862 podStartE2EDuration="2.614858866s" totalImagesPullingTime="895.454004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:58.613170712 +0000 UTC m=+2877.129666487" watchObservedRunningTime="2026-11-02 22:22:58.614858866 +0000 UTC m=+2877.131354639"
Nov 02 22:23:02.862104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:02.862069    5580 scope.go:122] "RemoveContainer" containerID="1a71652067e5238f4e17a4b2f6a397c4f9571d27daf82fb16aa88d3a621fa6e2"
Nov 02 22:23:09.507697 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:09.507659    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:23:19.761059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.761016    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"]
Nov 02 22:23:19.766587 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.766556    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.795775 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.795725    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796012 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.795830    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97743216-5aa5-42fb-a341-718c72c14b4f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796012 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.795940    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz2bm\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-kube-api-access-sz2bm\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796010    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97743216-5aa5-42fb-a341-718c72c14b4f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796061    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796151    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796189    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796217    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.796325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.796253    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.811426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.811384    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"]
Nov 02 22:23:19.814970 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.814932    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"]
Nov 02 22:23:19.815156 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.815079    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.818500 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.818474    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.859090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.859060    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qzh4z\""
Nov 02 22:23:19.865431 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.865397    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"]
Nov 02 22:23:19.897220 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897184    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.897220 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897230    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897309    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897357    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.897469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897377    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897394    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897469 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897457    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.897680 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897481    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.897680 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897506    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.897680 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897607    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.897828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897692    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897727    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.897940 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897866    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.897940 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897906    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897937    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97743216-5aa5-42fb-a341-718c72c14b4f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.897986    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898007    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.898095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898037    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.898095 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898072    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898097    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfznw\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-kube-api-access-qfznw\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898147    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898197    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898236    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898239    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.898338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898290    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.898619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898347    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.898619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898380    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz2bm\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-kube-api-access-sz2bm\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898408    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fsp7\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-kube-api-access-4fsp7\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:19.898619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898510    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97743216-5aa5-42fb-a341-718c72c14b4f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898619 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898591    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.898829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898626    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97743216-5aa5-42fb-a341-718c72c14b4f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.898829 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.898653    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dfaaa24-2c65-4d37-96e2-82b16948e324-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.900002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.899979    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97743216-5aa5-42fb-a341-718c72c14b4f-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.900928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.900910    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97743216-5aa5-42fb-a341-718c72c14b4f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.917941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.917904    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"]
Nov 02 22:23:19.954806 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.954763    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.955002 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.954874    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz2bm\" (UniqueName: \"kubernetes.io/projected/97743216-5aa5-42fb-a341-718c72c14b4f-kube-api-access-sz2bm\") pod \"maas-default-gateway-openshift-default-65c448b547-vdlkp\" (UID: \"97743216-5aa5-42fb-a341-718c72c14b4f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:19.980822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.980781    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"]
Nov 02 22:23:19.999939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.999899    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:19.999939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:19.999948    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fsp7\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-kube-api-access-4fsp7\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000015    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000054    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dfaaa24-2c65-4d37-96e2-82b16948e324-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000106    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000181    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000217    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000249    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000280    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000312    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000342    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000394    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000424    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000463    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000487    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfznw\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-kube-api-access-qfznw\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.000535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000526    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.001028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000541    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.001028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000571    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.001028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000598    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.001028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.000936    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.001260 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.001193    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.003935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.001776    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.003935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.001834    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dfaaa24-2c65-4d37-96e2-82b16948e324-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.003935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.002335    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.003935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.002382    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.003935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.002385    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.004291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.003938    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.004291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.004023    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.004291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.004155    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.004291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.004190    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.004291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.004207    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.006106 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.006087    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.058614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.058528    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.058614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.058549    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.058614 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.058601    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfznw\" (UniqueName: \"kubernetes.io/projected/0dfaaa24-2c65-4d37-96e2-82b16948e324-kube-api-access-qfznw\") pod \"maas-default-gateway-openshift-default-65c448b547-v7sg6\" (UID: \"0dfaaa24-2c65-4d37-96e2-82b16948e324\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.059087 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.059068    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fsp7\" (UniqueName: \"kubernetes.io/projected/5fdd3a32-f031-438c-a5a5-cae4b101b2e1-kube-api-access-4fsp7\") pod \"maas-default-gateway-openshift-default-65c448b547-trltt\" (UID: \"5fdd3a32-f031-438c-a5a5-cae4b101b2e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.079439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.079403    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:20.133763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.132899    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:20.141908 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.140695    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:20.505359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.505319    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"]
Nov 02 22:23:20.547903 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:23:20.547869    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97743216_5aa5_42fb_a341_718c72c14b4f.slice/crio-6ded0c40a20bad0b9e742004ad3b4bbb2efaac3e9629475d932fc4e41a1d71be WatchSource:0}: Error finding container 6ded0c40a20bad0b9e742004ad3b4bbb2efaac3e9629475d932fc4e41a1d71be: Status 404 returned error can't find the container with id 6ded0c40a20bad0b9e742004ad3b4bbb2efaac3e9629475d932fc4e41a1d71be
Nov 02 22:23:20.550085 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.550051    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.550157 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.550124    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.550191 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.550157    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.560016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.559986    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"]
Nov 02 22:23:20.562595 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:23:20.562485    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfaaa24_2c65_4d37_96e2_82b16948e324.slice/crio-c229265a95a313fc80d87ad35239627c859d70c6cd0664a7acf9238e531ef211 WatchSource:0}: Error finding container c229265a95a313fc80d87ad35239627c859d70c6cd0664a7acf9238e531ef211: Status 404 returned error can't find the container with id c229265a95a313fc80d87ad35239627c859d70c6cd0664a7acf9238e531ef211
Nov 02 22:23:20.565009 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.564948    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.565126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.565063    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.565126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.565107    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.606617 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.606501    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" event={"ID":"0dfaaa24-2c65-4d37-96e2-82b16948e324","Type":"ContainerStarted","Data":"c229265a95a313fc80d87ad35239627c859d70c6cd0664a7acf9238e531ef211"}
Nov 02 22:23:20.607075 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.607052    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"]
Nov 02 22:23:20.608652 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.608629    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" event={"ID":"97743216-5aa5-42fb-a341-718c72c14b4f","Type":"ContainerStarted","Data":"6ded0c40a20bad0b9e742004ad3b4bbb2efaac3e9629475d932fc4e41a1d71be"}
Nov 02 22:23:20.611038 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:23:20.611007    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdd3a32_f031_438c_a5a5_cae4b101b2e1.slice/crio-d0f83214fd5c6953d2599d519c855409dcdb8b02c167680e95a6bb533a3825f4 WatchSource:0}: Error finding container d0f83214fd5c6953d2599d519c855409dcdb8b02c167680e95a6bb533a3825f4: Status 404 returned error can't find the container with id d0f83214fd5c6953d2599d519c855409dcdb8b02c167680e95a6bb533a3825f4
Nov 02 22:23:20.613356 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.613320    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.613450 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.613391    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:20.613450 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:20.613421    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:23:21.669994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.669935    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" event={"ID":"0dfaaa24-2c65-4d37-96e2-82b16948e324","Type":"ContainerStarted","Data":"da491d89715309ce45ab5059b927d9ee2b684c87bffacf41a34f81f69344dda6"}
Nov 02 22:23:21.672277 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.672234    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" event={"ID":"5fdd3a32-f031-438c-a5a5-cae4b101b2e1","Type":"ContainerStarted","Data":"d1f95e29420044dc421ec68872c998142ea6fb838600a51f0627a0fd5d2b12b7"}
Nov 02 22:23:21.672434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.672307    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" event={"ID":"5fdd3a32-f031-438c-a5a5-cae4b101b2e1","Type":"ContainerStarted","Data":"d0f83214fd5c6953d2599d519c855409dcdb8b02c167680e95a6bb533a3825f4"}
Nov 02 22:23:21.673949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.673923    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" event={"ID":"97743216-5aa5-42fb-a341-718c72c14b4f","Type":"ContainerStarted","Data":"2ad1f759b2b704e72cdaf8a3bd14b3722db5f2cd0455459c483ae85fd8c74d99"}
Nov 02 22:23:21.861596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.861538    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podStartSLOduration=2.861524376 podStartE2EDuration="2.861524376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:23:21.853722171 +0000 UTC m=+2900.370217967" watchObservedRunningTime="2026-11-02 22:23:21.861524376 +0000 UTC m=+2900.378020149"
Nov 02 22:23:21.971417 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:21.971308    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podStartSLOduration=2.97128889 podStartE2EDuration="2.97128889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:23:21.970851758 +0000 UTC m=+2900.487347533" watchObservedRunningTime="2026-11-02 22:23:21.97128889 +0000 UTC m=+2900.487784646"
Nov 02 22:23:22.090983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.080349    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:22.090983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.084107    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:22.090983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.084174    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:22.090983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.088137    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podStartSLOduration=3.088115389 podStartE2EDuration="3.088115389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:23:22.086469263 +0000 UTC m=+2900.602965039" watchObservedRunningTime="2026-11-02 22:23:22.088115389 +0000 UTC m=+2900.604611162"
Nov 02 22:23:22.137984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.137784    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:22.137984 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.137938    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:22.138271 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.138005    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:22.152983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.142152    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:22.152983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.148099    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:22.152983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:22.148167    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:23.084981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.080378    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:23.084981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.080445    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:23.138113 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.138072    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:23.138372 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.138346    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:23.143376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.143339    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:23.143663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:23.143632    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:24.086989 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.083470    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:24.086989 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.083551    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:24.153166 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.153107    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:24.153502 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.153456    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:24.153737 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.153719    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:24.153889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:24.153871    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:25.086491 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.086443    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:25.086843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.086804    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:25.135438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.135396    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:25.136090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.136053    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:25.147675 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.147636    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:25.148022 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:25.147939    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:26.090982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.089393    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:26.090982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.089461    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:26.137005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.136458    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:26.137005 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.136532    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:26.144972 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.144914    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:26.145238 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:26.145209    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:26.369215 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:23:26.369116    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:23:27.080410 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.080362    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:27.080691 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.080438    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:27.133947 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.133902    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:27.134151 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.134008    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:27.142292 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.142251    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:27.142670 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:27.142324    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:28.080435 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.080386    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Nov 02 22:23:28.080676 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.080484    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Nov 02 22:23:28.134059 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.134004    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Nov 02 22:23:28.134346 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.134095    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Nov 02 22:23:28.142146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.142102    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Nov 02 22:23:28.142337 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:28.142170    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Nov 02 22:23:30.080566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.080519    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:30.081165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.080609    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:30.134230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.134181    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:30.134470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.134268    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:30.182219 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.182173    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:30.182536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:30.182498    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:31.135983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.134158    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:31.135983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.134230    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:31.179983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.176182    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:31.179983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.176259    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:31.229151 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.229091    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:31.229493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.229438    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:31.735736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.735697    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:31.737336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.737310    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:31.738556 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.738535    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6"
Nov 02 22:23:31.963655 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.963601    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:23:31.963881 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:31.963851    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" podUID="7e5eaae0-697a-4cf3-87fd-7f755111e817" containerName="manager" containerID="cri-o://128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9" gracePeriod=30
Nov 02 22:23:32.182880 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.182257    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:32.182880 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.182326    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:32.228135 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.228036    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:23:32.228927 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.228383    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:23:32.257268 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.257113    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:32.258139 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.258003    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:32.261994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.261951    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp"
Nov 02 22:23:32.298017 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.297864    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:23:32.353553 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.353510    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\") pod \"7e5eaae0-697a-4cf3-87fd-7f755111e817\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") "
Nov 02 22:23:32.353736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.353567    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\" (UniqueName: \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\") pod \"7e5eaae0-697a-4cf3-87fd-7f755111e817\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") "
Nov 02 22:23:32.353736 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.353614    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\") pod \"7e5eaae0-697a-4cf3-87fd-7f755111e817\" (UID: \"7e5eaae0-697a-4cf3-87fd-7f755111e817\") "
Nov 02 22:23:32.356025 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.355984    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh" pod "7e5eaae0-697a-4cf3-87fd-7f755111e817" (UID: "7e5eaae0-697a-4cf3-87fd-7f755111e817"). InnerVolumeSpecName "kube-api-access-f9mxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:23:32.356326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.356304    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs" pod "7e5eaae0-697a-4cf3-87fd-7f755111e817" (UID: "7e5eaae0-697a-4cf3-87fd-7f755111e817"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:23:32.356403 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.356327    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert" pod "7e5eaae0-697a-4cf3-87fd-7f755111e817" (UID: "7e5eaae0-697a-4cf3-87fd-7f755111e817"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:23:32.434788 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.434747    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:32.435125 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.435105    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:32.436063 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.436046    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt"
Nov 02 22:23:32.455521 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.455493    5580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-webhook-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:23:32.455651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.455527    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9mxh\" (UniqueName: \"kubernetes.io/projected/7e5eaae0-697a-4cf3-87fd-7f755111e817-kube-api-access-f9mxh\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:23:32.455651 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.455546    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e5eaae0-697a-4cf3-87fd-7f755111e817-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:23:32.818790 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.818753    5580 generic.go:356] "Generic (PLEG): container finished" podID="7e5eaae0-697a-4cf3-87fd-7f755111e817" containerID="128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9" exitCode=0
Nov 02 22:23:32.818985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.818803    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7"
Nov 02 22:23:32.818985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.818844    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" event={"ID":"7e5eaae0-697a-4cf3-87fd-7f755111e817","Type":"ContainerDied","Data":"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"}
Nov 02 22:23:32.818985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.818895    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-pbmv7" event={"ID":"7e5eaae0-697a-4cf3-87fd-7f755111e817","Type":"ContainerDied","Data":"00bf4dcd44697c0b185ecc7bf58e10fa425b775f5181b14e6341a8aa7a41a1bb"}
Nov 02 22:23:32.818985 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.818918    5580 scope.go:122] "RemoveContainer" containerID="128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"
Nov 02 22:23:32.829702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.829378    5580 scope.go:122] "RemoveContainer" containerID="128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"
Nov 02 22:23:32.830039 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:23:32.830011    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9\": container with ID starting with 128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9 not found: ID does not exist" containerID="128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"
Nov 02 22:23:32.830214 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.830053    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9"} err="failed to get container status \"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9\": rpc error: code = NotFound desc = could not find container \"128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9\": container with ID starting with 128a348755bbcf813351456ef552ab8ebcea5a2593398e13db23601abe5986c9 not found: ID does not exist"
Nov 02 22:23:32.982790 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.982746    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:23:32.985286 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:32.985249    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-pbmv7"]
Nov 02 22:23:33.592743 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:33.592697    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e5eaae0-697a-4cf3-87fd-7f755111e817" path="/var/lib/kubelet/pods/7e5eaae0-697a-4cf3-87fd-7f755111e817/volumes"
Nov 02 22:23:34.286729 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.286687    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-j4bwf"]
Nov 02 22:23:34.287134 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.287120    5580 cpu_manager.go:414] "containerMap: removing container" podUID="7e5eaae0-697a-4cf3-87fd-7f755111e817" containerName="manager"
Nov 02 22:23:34.287179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.287135    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e5eaae0-697a-4cf3-87fd-7f755111e817" containerName="manager"
Nov 02 22:23:34.287219 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.287210    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e5eaae0-697a-4cf3-87fd-7f755111e817" containerName="manager"
Nov 02 22:23:34.290404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.290389    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.342568 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.342531    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 22:23:34.376773 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.376734    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-webhook-cert\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.376949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.376787    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-metrics-certs\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.376949 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.376933    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92h6s\" (UniqueName: \"kubernetes.io/projected/3065b811-5eab-4780-bbb7-8302e87bb1b8-kube-api-access-92h6s\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.392393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.392356    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-b76mn\""
Nov 02 22:23:34.392580 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.392362    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 22:23:34.392580 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.392391    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 22:23:34.394153 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.394121    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-j4bwf"]
Nov 02 22:23:34.477923 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.477877    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-metrics-certs\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.478173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.478017    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92h6s\" (UniqueName: \"kubernetes.io/projected/3065b811-5eab-4780-bbb7-8302e87bb1b8-kube-api-access-92h6s\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.478173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.478143    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-webhook-cert\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.480812 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.480785    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-webhook-cert\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.480912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.480785    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3065b811-5eab-4780-bbb7-8302e87bb1b8-metrics-certs\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.534843 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.534808    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92h6s\" (UniqueName: \"kubernetes.io/projected/3065b811-5eab-4780-bbb7-8302e87bb1b8-kube-api-access-92h6s\") pod \"maas-controller-5cc9f7769b-j4bwf\" (UID: \"3065b811-5eab-4780-bbb7-8302e87bb1b8\") " pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.603203 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.603089    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:34.973837 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:34.973810    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cc9f7769b-j4bwf"]
Nov 02 22:23:34.975930 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:23:34.975888    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3065b811_5eab_4780_bbb7_8302e87bb1b8.slice/crio-87339ea6b75e3db61f84bf972741b9b39d1b4eba8731229b8973ada4c45cf4c0 WatchSource:0}: Error finding container 87339ea6b75e3db61f84bf972741b9b39d1b4eba8731229b8973ada4c45cf4c0: Status 404 returned error can't find the container with id 87339ea6b75e3db61f84bf972741b9b39d1b4eba8731229b8973ada4c45cf4c0
Nov 02 22:23:35.841108 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:35.841069    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf" event={"ID":"3065b811-5eab-4780-bbb7-8302e87bb1b8","Type":"ContainerStarted","Data":"87339ea6b75e3db61f84bf972741b9b39d1b4eba8731229b8973ada4c45cf4c0"}
Nov 02 22:23:35.841566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:35.841199    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:36.002229 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:36.002174    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf" podStartSLOduration=1.200013713 podStartE2EDuration="2.002158201s" totalImagesPullingTime="802.144488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:23:36.00119327 +0000 UTC m=+2914.517689044" watchObservedRunningTime="2026-11-02 22:23:36.002158201 +0000 UTC m=+2914.518653992"
Nov 02 22:23:36.848245 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:36.848211    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf" event={"ID":"3065b811-5eab-4780-bbb7-8302e87bb1b8","Type":"ContainerStarted","Data":"e6a7e07120a80249af5eb4950ec6e17838a78399cd0da08b6e301089c057a58c"}
Nov 02 22:23:47.854718 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:23:47.854681    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cc9f7769b-j4bwf"
Nov 02 22:23:56.311754 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:23:56.311708    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:24:19.347030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:19.346990    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:24:26.311724 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:24:26.311682    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:24:35.330693 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.330654    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"]
Nov 02 22:24:35.334021 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.333997    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.437500 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.437457    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"]
Nov 02 22:24:35.454094 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454041    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454101    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454181    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454213    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454297 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454237    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454410    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454483    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454596 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454576    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgbp9\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-kube-api-access-hgbp9\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.454643 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.454632    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556064    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgbp9\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-kube-api-access-hgbp9\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556112    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556144    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556170    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556200    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556218    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556251    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556316    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556387 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556350    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556630    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-credential-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556715    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-socket\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556747    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-workload-certs\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.556853 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.556791    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-data\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.557122 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.557103    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.558664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.558635    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-envoy\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.559074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.559053    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.612926 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.612834    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-istio-token\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.613123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.613025    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgbp9\" (UniqueName: \"kubernetes.io/projected/95921fe2-1e3d-4fdd-a833-d4fbef78da76-kube-api-access-hgbp9\") pod \"maas-default-gateway-openshift-default-65c448b547-dnxdq\" (UID: \"95921fe2-1e3d-4fdd-a833-d4fbef78da76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:35.646976 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:35.646905    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:36.070274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.070244    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"]
Nov 02 22:24:36.073048 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:24:36.073010    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95921fe2_1e3d_4fdd_a833_d4fbef78da76.slice/crio-cd00a679db2bca7124da55e5ace4883cd073175f014573cc2c062fb968ac487b WatchSource:0}: Error finding container cd00a679db2bca7124da55e5ace4883cd073175f014573cc2c062fb968ac487b: Status 404 returned error can't find the container with id cd00a679db2bca7124da55e5ace4883cd073175f014573cc2c062fb968ac487b
Nov 02 22:24:36.075237 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.075203    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:24:36.075325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.075276    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:24:36.075325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.075305    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:24:36.143561 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.143518    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" event={"ID":"95921fe2-1e3d-4fdd-a833-d4fbef78da76","Type":"ContainerStarted","Data":"a4a5c5ea6616f9fec63d2e6eaf38f924739d56c7154379764668ec431fc798e0"}
Nov 02 22:24:36.143787 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.143588    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" event={"ID":"95921fe2-1e3d-4fdd-a833-d4fbef78da76","Type":"ContainerStarted","Data":"cd00a679db2bca7124da55e5ace4883cd073175f014573cc2c062fb968ac487b"}
Nov 02 22:24:36.261115 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.261059    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podStartSLOduration=1.261043453 podStartE2EDuration="1.261043453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:24:36.259252172 +0000 UTC m=+2974.775747939" watchObservedRunningTime="2026-11-02 22:24:36.261043453 +0000 UTC m=+2974.777539226"
Nov 02 22:24:36.647516 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:36.647482    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:37.649662 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:37.649623    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:37.650102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:37.649684    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:38.647987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:38.647919    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:38.648184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:38.648013    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:39.648107 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:39.648070    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:39.648522 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:39.648128    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:40.619269 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:40.619231    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:24:40.648327 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:40.648279    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:40.648839 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:40.648346    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:41.648343 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:41.648272    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:41.648779 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:41.648377    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:42.648102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:42.648059    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:42.648326 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:42.648121    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:43.648493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:43.648448    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Nov 02 22:24:43.648937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:43.648529    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Nov 02 22:24:45.690092 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:45.690055    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:45.690613 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:45.690581    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:24:46.738151 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:46.738099    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:46.738846 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:46.738806    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:24:47.745184 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:47.745120    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:47.745872 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:47.745834    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:24:47.785162 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:47.785106    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:47.785535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:47.785502    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:24:48.787195 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:48.787150    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:24:48.787749 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:48.787221    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:24:49.487264 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:49.487229    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:49.487618 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:49.487594    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:49.488405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:49.488376    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq"
Nov 02 22:24:50.927393 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:24:50.927353    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:24:56.310266 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:24:56.310219    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:25:02.256070 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:02.256041    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:25:02.265709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:02.265674    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:25:02.267606 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:02.267581    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:25:02.276292 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:02.276269    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:25:11.851637 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:11.851594    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:25:26.311750 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:25:26.311701    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:25:30.405929 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:30.405893    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:25:32.786986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:32.786151    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-v7sg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:25:32.786986 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:32.786275    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-v7sg6" podUID="0dfaaa24-2c65-4d37-96e2-82b16948e324" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:25:33.308004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:33.304094    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:25:33.308004 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:33.304173    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:25:33.488123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:33.488078    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:25:33.488428 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:25:33.488393    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:25:56.310838 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:25:56.310797    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:26:02.135311 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:26:02.135260    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:26:12.734454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:26:12.734370    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:26:24.834709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:26:24.834659    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:26:26.310862 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:26:26.310823    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:26:47.443721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:26:47.443683    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:26:56.311718 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:26:56.311672    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:26:56.538200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:26:56.538159    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:27:15.050071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:15.050030    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"]
Nov 02 22:27:15.099633 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:15.099591    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36900405-g676f"]
Nov 02 22:27:15.348505 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:15.348406    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:27:15.592975 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:15.592931    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e03df3c-4de5-4122-85c0-deb318fc57d3" path="/var/lib/kubelet/pods/0e03df3c-4de5-4122-85c0-deb318fc57d3/volumes"
Nov 02 22:27:26.311902 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:27:26.311856    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:27:27.344090 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:27.344047    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:27:30.046503 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:30.046447    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"]
Nov 02 22:27:30.096091 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:30.096047    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-42abe6b5-8q8qt"]
Nov 02 22:27:31.594254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:31.594217    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a9d736f-5af8-42bd-9853-ba104020c1fa" path="/var/lib/kubelet/pods/5a9d736f-5af8-42bd-9853-ba104020c1fa/volumes"
Nov 02 22:27:47.539259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:47.539214    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:27:50.532244 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:50.532199    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:50.533639 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:27:50.533598    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:56.311077 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:27:56.311037    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:28:09.623803 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:28:09.623765    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:28:22.841826 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:28:22.841788    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:28:26.311837 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:28:26.311799    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:28:55.130873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:28:55.130833    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:28:56.334495 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:28:56.334444    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:29:02.940429 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:29:02.940396    5580 scope.go:122] "RemoveContainer" containerID="1277bf14d52cf82c1a28f68c029e79eb877e83d1cfb9b7d4c049b3487397704e"
Nov 02 22:29:02.950443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:29:02.950421    5580 scope.go:122] "RemoveContainer" containerID="8b212ffb4f8cd6725d0915db12ad8a47fc2a6a3a844e3ba005a3190aa9c703a1"
Nov 02 22:29:16.554412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:29:16.554328    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:29:26.311199 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:29:26.311158    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:29:27.217894 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:29:27.217857    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:29:56.310759 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:29:56.310717    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:30:00.156499 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.156464    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:00.160517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.160494    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.195250 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.195211    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fsxb\" (UniqueName: \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.195448 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.195369    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.213142 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.213109    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c4p9p\""
Nov 02 22:30:00.256874 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.256835    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:00.296556 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.296519    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fsxb\" (UniqueName: \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.296751 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.296597    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.297336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.297312    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.353052 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.353012    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fsxb\" (UniqueName: \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\") pod \"maas-api-key-cleanup-29894310-tbb5v\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.479292 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.479259    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:00.852329 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:00.852305    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:00.855120 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:30:00.855092    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dd81fd_66fc_418d_b5e9_f5b5524aaaf0.slice/crio-459b47b876d06f8b01419bbdcd2277b5d57affc00a53f4adb96e729e56dcba6c WatchSource:0}: Error finding container 459b47b876d06f8b01419bbdcd2277b5d57affc00a53f4adb96e729e56dcba6c: Status 404 returned error can't find the container with id 459b47b876d06f8b01419bbdcd2277b5d57affc00a53f4adb96e729e56dcba6c
Nov 02 22:30:01.138684 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:01.138577    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerStarted","Data":"459b47b876d06f8b01419bbdcd2277b5d57affc00a53f4adb96e729e56dcba6c"}
Nov 02 22:30:02.145212 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.145126    5580 generic.go:356] "Generic (PLEG): container finished" podID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerID="39948dda9257552c3af3d61d7d8ffed8dced64dab5243623097f33d8f0ddfa11" exitCode=60
Nov 02 22:30:02.145621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.145207    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerDied","Data":"39948dda9257552c3af3d61d7d8ffed8dced64dab5243623097f33d8f0ddfa11"}
Nov 02 22:30:02.145621 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.145435    5580 scope.go:122] "RemoveContainer" containerID="39948dda9257552c3af3d61d7d8ffed8dced64dab5243623097f33d8f0ddfa11"
Nov 02 22:30:02.291205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.291179    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:30:02.300831 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.300797    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:30:02.302518 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.302495    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:30:02.320102 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:02.320073    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:30:03.152873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:03.152836    5580 generic.go:356] "Generic (PLEG): container finished" podID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerID="bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677" exitCode=60
Nov 02 22:30:03.153409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:03.152921    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerDied","Data":"bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677"}
Nov 02 22:30:03.153409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:03.153032    5580 scope.go:122] "RemoveContainer" containerID="39948dda9257552c3af3d61d7d8ffed8dced64dab5243623097f33d8f0ddfa11"
Nov 02 22:30:03.153409 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:03.153318    5580 scope.go:122] "RemoveContainer" containerID="bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677"
Nov 02 22:30:03.153646 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:30:03.153624    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894310-tbb5v_odh-ai-gateway-infra(c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0"
Nov 02 22:30:04.160979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:04.160914    5580 scope.go:122] "RemoveContainer" containerID="bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677"
Nov 02 22:30:04.161396 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:30:04.161183    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894310-tbb5v_odh-ai-gateway-infra(c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0"
Nov 02 22:30:06.131607 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:06.131570    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:30:14.587038 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:14.586998    5580 scope.go:122] "RemoveContainer" containerID="bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677"
Nov 02 22:30:15.218386 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:15.218353    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerStarted","Data":"a6660861e8d545ad395ec5232e6594ba2cc2f01ffca02331f1e66a2add73b74c"}
Nov 02 22:30:15.331827 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:15.331766    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" podStartSLOduration=15.3317498 podStartE2EDuration="15.3317498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:15.331107973 +0000 UTC m=+3313.847603760" watchObservedRunningTime="2026-11-02 22:30:15.3317498 +0000 UTC m=+3313.848245570"
Nov 02 22:30:16.226019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:16.225982    5580 generic.go:356] "Generic (PLEG): container finished" podID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerID="a6660861e8d545ad395ec5232e6594ba2cc2f01ffca02331f1e66a2add73b74c" exitCode=60
Nov 02 22:30:16.226464 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:16.226032    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerDied","Data":"a6660861e8d545ad395ec5232e6594ba2cc2f01ffca02331f1e66a2add73b74c"}
Nov 02 22:30:16.226464 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:16.226100    5580 scope.go:122] "RemoveContainer" containerID="bb6b48d74a8869b0bb49f9fab5d7028c4d5fa9adcf9472db1faa8b4a5e114677"
Nov 02 22:30:16.226464 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:16.226445    5580 scope.go:122] "RemoveContainer" containerID="a6660861e8d545ad395ec5232e6594ba2cc2f01ffca02331f1e66a2add73b74c"
Nov 02 22:30:16.226741 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:30:16.226718    5580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29894310-tbb5v_odh-ai-gateway-infra(c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0"
Nov 02 22:30:16.340475 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:16.340435    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:17.376218 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.376188    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:17.499224 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.499144    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\") pod \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") "
Nov 02 22:30:17.499401 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.499326    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\" (UniqueName: \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\") pod \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\" (UID: \"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0\") "
Nov 02 22:30:17.499523 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.499497    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca" pod "c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" (UID: "c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:17.499712 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.499692    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-maas-api-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:30:17.501426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.501402    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb" pod "c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" (UID: "c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0"). InnerVolumeSpecName "kube-api-access-2fsxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:17.601603 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:17.601560    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fsxb\" (UniqueName: \"kubernetes.io/projected/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0-kube-api-access-2fsxb\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:30:18.238440 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:18.238401    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v" event={"ID":"c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0","Type":"ContainerDied","Data":"459b47b876d06f8b01419bbdcd2277b5d57affc00a53f4adb96e729e56dcba6c"}
Nov 02 22:30:18.238664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:18.238465    5580 scope.go:122] "RemoveContainer" containerID="a6660861e8d545ad395ec5232e6594ba2cc2f01ffca02331f1e66a2add73b74c"
Nov 02 22:30:18.238664 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:18.238408    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"
Nov 02 22:30:18.400946 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:18.400901    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:18.404352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:18.404320    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894310-tbb5v"]
Nov 02 22:30:19.593036 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:19.593003    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" path="/var/lib/kubelet/pods/c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0/volumes"
Nov 02 22:30:24.615341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:24.615304    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:30:26.310809 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:30:26.310772    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:30:50.637580 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:30:50.637530    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:30:56.312847 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:30:56.312804    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:31:19.946382 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:31:19.946342    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:31:26.310950 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:31:26.310911    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:31:38.046975 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:31:38.046922    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:31:56.311462 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:31:56.311421    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:32:26.310979 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:32:26.310924    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:32:56.311107 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:32:56.311068    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:33:26.311924 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:33:26.311880    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:33:26.913678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:33:26.913639    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:33:44.319447 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:33:44.319404    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:33:53.527615 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:33:53.527574    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:33:56.311250 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:33:56.311207    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:34:13.430931 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:13.430889    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:34:23.737900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.737855    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:34:23.738356 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738340    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738358    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738369    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738374    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738445    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738453    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.738504 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.738460    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:34:23.741577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.741559    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.793840 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.793807    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-8b2f0b-serving-cert\""
Nov 02 22:34:23.814247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.814212    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.814438 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.814356    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.814493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.814447    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.814535 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.814502    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvq8j\" (UniqueName: \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.816398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.816377    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:34:23.915582 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.915544    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.915771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.915648    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.915771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.915693    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.915771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.915743    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvq8j\" (UniqueName: \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.915977 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:34:23.915790    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-serving-cert: secret "maas-api-e2e-worker-single-main-8b2f0b-serving-cert" not found
Nov 02 22:34:23.915977 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:34:23.915883    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls podName:930eec38-0164-4a2a-a756-6a948cefbd97 nodeName:}" failed. No retries permitted until 2026-11-02 22:34:24.415857409 +0000 UTC m=+3562.932353176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls") pod "maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" (UID: "930eec38-0164-4a2a-a756-6a948cefbd97") : secret "maas-api-e2e-worker-single-main-8b2f0b-serving-cert" not found
Nov 02 22:34:23.916301 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.916281    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.918352 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.918328    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:23.972987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:23.972923    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvq8j\" (UniqueName: \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:24.420941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:24.420897    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:24.423647 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:24.423619    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:24.654841 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:24.654803    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:25.029696 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:25.029668    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:34:25.032554 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:34:25.032525    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930eec38_0164_4a2a_a756_6a948cefbd97.slice/crio-4e38639bd2bb8cb05bf8980b186ed4b0e5b99ecf57ec2f486ff837418f522605 WatchSource:0}: Error finding container 4e38639bd2bb8cb05bf8980b186ed4b0e5b99ecf57ec2f486ff837418f522605: Status 404 returned error can't find the container with id 4e38639bd2bb8cb05bf8980b186ed4b0e5b99ecf57ec2f486ff837418f522605
Nov 02 22:34:25.033889 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:25.033868    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:34:25.544360 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:25.544322    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" event={"ID":"930eec38-0164-4a2a-a756-6a948cefbd97","Type":"ContainerStarted","Data":"4e38639bd2bb8cb05bf8980b186ed4b0e5b99ecf57ec2f486ff837418f522605"}
Nov 02 22:34:26.311467 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:34:26.311378    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:34:26.551817 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:26.551784    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" event={"ID":"930eec38-0164-4a2a-a756-6a948cefbd97","Type":"ContainerStarted","Data":"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"}
Nov 02 22:34:26.552028 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:26.551905    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:26.670725 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:26.670652    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" podStartSLOduration=2.756061041 podStartE2EDuration="3.670638266s" totalImagesPullingTime="914.577225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:34:26.669111524 +0000 UTC m=+3565.185607311" watchObservedRunningTime="2026-11-02 22:34:26.670638266 +0000 UTC m=+3565.187134040"
Nov 02 22:34:31.857385 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:31.857343    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:34:32.563946 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:32.563915    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:34:50.751412 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:34:50.751372    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:34:56.311038 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:34:56.310952    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:35:02.326402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:02.326370    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:35:02.337558 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:02.337529    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:35:02.345248 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:02.345226    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:35:02.355743 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:02.355713    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:35:15.953123 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:15.953046    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:35:26.311381 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:35:26.311334    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:35:44.010979 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:35:44.010928    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:35:56.310423 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:35:56.310380    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:36:00.045083 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:00.045042    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:36:00.045511 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:00.045316    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" podUID="930eec38-0164-4a2a-a756-6a948cefbd97" containerName="maas-api" containerID="cri-o://5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c" gracePeriod=30
Nov 02 22:36:01.299124 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.299091    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:36:01.466867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.466826    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\") pod \"930eec38-0164-4a2a-a756-6a948cefbd97\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") "
Nov 02 22:36:01.467135 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.466883    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\" (UniqueName: \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\") pod \"930eec38-0164-4a2a-a756-6a948cefbd97\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") "
Nov 02 22:36:01.467135 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.467015    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") pod \"930eec38-0164-4a2a-a756-6a948cefbd97\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") "
Nov 02 22:36:01.467286 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.467154    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\") pod \"930eec38-0164-4a2a-a756-6a948cefbd97\" (UID: \"930eec38-0164-4a2a-a756-6a948cefbd97\") "
Nov 02 22:36:01.467286 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.467259    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca" pod "930eec38-0164-4a2a-a756-6a948cefbd97" (UID: "930eec38-0164-4a2a-a756-6a948cefbd97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:36:01.467540 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.467522    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/930eec38-0164-4a2a-a756-6a948cefbd97-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:36:01.469226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.469203    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs" pod "930eec38-0164-4a2a-a756-6a948cefbd97" (UID: "930eec38-0164-4a2a-a756-6a948cefbd97"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:36:01.469338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.469221    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls" pod "930eec38-0164-4a2a-a756-6a948cefbd97" (UID: "930eec38-0164-4a2a-a756-6a948cefbd97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:36:01.469338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.469222    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j" pod "930eec38-0164-4a2a-a756-6a948cefbd97" (UID: "930eec38-0164-4a2a-a756-6a948cefbd97"). InnerVolumeSpecName "kube-api-access-vvq8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:36:01.568721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.568682    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:36:01.568721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.568715    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvq8j\" (UniqueName: \"kubernetes.io/projected/930eec38-0164-4a2a-a756-6a948cefbd97-kube-api-access-vvq8j\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:36:01.568721 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:01.568729    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/930eec38-0164-4a2a-a756-6a948cefbd97-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:36:02.031318 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.031282    5580 generic.go:356] "Generic (PLEG): container finished" podID="930eec38-0164-4a2a-a756-6a948cefbd97" containerID="5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c" exitCode=0
Nov 02 22:36:02.031517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.031342    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"
Nov 02 22:36:02.031517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.031359    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" event={"ID":"930eec38-0164-4a2a-a756-6a948cefbd97","Type":"ContainerDied","Data":"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"}
Nov 02 22:36:02.031517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.031437    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m" event={"ID":"930eec38-0164-4a2a-a756-6a948cefbd97","Type":"ContainerDied","Data":"4e38639bd2bb8cb05bf8980b186ed4b0e5b99ecf57ec2f486ff837418f522605"}
Nov 02 22:36:02.031517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.031478    5580 scope.go:122] "RemoveContainer" containerID="5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"
Nov 02 22:36:02.040780 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.040761    5580 scope.go:122] "RemoveContainer" containerID="5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"
Nov 02 22:36:02.041111 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:36:02.041089    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c\": container with ID starting with 5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c not found: ID does not exist" containerID="5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"
Nov 02 22:36:02.041171 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.041121    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c"} err="failed to get container status \"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c\": rpc error: code = NotFound desc = could not find container \"5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c\": container with ID starting with 5675b69398b7e990f11f5be0a48eaae89d787bd7d808692f16c029d0f95b861c not found: ID does not exist"
Nov 02 22:36:02.193912 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.193872    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:36:02.243708 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:02.243672    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8b2f0b-764955c646-ljl6m"]
Nov 02 22:36:03.592699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:03.592663    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="930eec38-0164-4a2a-a756-6a948cefbd97" path="/var/lib/kubelet/pods/930eec38-0164-4a2a-a756-6a948cefbd97/volumes"
Nov 02 22:36:04.555063 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:04.555020    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:36:19.377234 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:36:19.377194    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:36:26.311896 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:36:26.311855    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:36:56.311862 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:36:56.311819    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:37:02.048715 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:02.048679    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:37:05.547982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:05.547187    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:37:05.547982 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:05.547271    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:37:26.310823 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:37:26.310780    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:37:32.828665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:32.828627    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:37:35.553983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:35.551142    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-dnxdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:37:35.553983 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:35.551245    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-dnxdq" podUID="95921fe2-1e3d-4fdd-a833-d4fbef78da76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:37:48.312046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:48.312010    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:37:56.309829 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:37:56.309786    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:37:59.460310 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:37:59.460276    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:38:11.871672 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:11.871592    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:38:22.521724 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:22.521685    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:38:26.311131 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:38:26.311083    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:38:42.224229 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:42.224191    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:38:49.542815 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.542774    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:38:49.543335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.543258    5580 cpu_manager.go:414] "containerMap: removing container" podUID="930eec38-0164-4a2a-a756-6a948cefbd97" containerName="maas-api"
Nov 02 22:38:49.543335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.543272    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="930eec38-0164-4a2a-a756-6a948cefbd97" containerName="maas-api"
Nov 02 22:38:49.543335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.543293    5580 cpu_manager.go:414] "containerMap: removing container" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:38:49.543335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.543298    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dd81fd-66fc-418d-b5e9-f5b5524aaaf0" containerName="cleanup"
Nov 02 22:38:49.543470 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.543366    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="930eec38-0164-4a2a-a756-6a948cefbd97" containerName="maas-api"
Nov 02 22:38:49.546569 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.546552    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.605771 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.601445    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ce65d173-openshift-default-dockercfg-7vqws\""
Nov 02 22:38:49.637821 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.637781    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.637837    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.637945    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.637993    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.638035    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96gh5\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638194 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.638172    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.638210    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638302 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.638278    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.638361 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.638322    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.652893 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.652861    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:38:49.739969 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.739914    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.739989    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740021    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96gh5\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740200 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740186    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740227    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740270    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740396 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740360    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740440    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740493    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740442    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740870 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740845    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.740930 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.740845    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.741048 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.741023    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.741119 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.741096    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.742783 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.742756    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.743019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.742997    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.801757 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.801680    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.801902 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.801827    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96gh5\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\") pod \"e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") " pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:49.862230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:49.860459    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:50.299342 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.299304    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:38:50.331388 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:38:50.331349    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bb4386_4c74_47c8_a894_358e64b747d0.slice/crio-3f618c4f78de8f1c726d1300721fed34987b4f3910738bf6eef3730d59b4e8fe WatchSource:0}: Error finding container 3f618c4f78de8f1c726d1300721fed34987b4f3910738bf6eef3730d59b4e8fe: Status 404 returned error can't find the container with id 3f618c4f78de8f1c726d1300721fed34987b4f3910738bf6eef3730d59b4e8fe
Nov 02 22:38:50.333652 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.333624    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:38:50.333749 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.333726    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:38:50.333802 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.333755    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:38:50.430763 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.430722    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"]
Nov 02 22:38:50.434426 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.434405    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.491720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.491666    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"]
Nov 02 22:38:50.549905 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.549732    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbx97\" (UniqueName: \"kubernetes.io/projected/be3ec110-72a1-428f-a093-9b6656b6d84e-kube-api-access-wbx97\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.550541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.549987    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be3ec110-72a1-428f-a093-9b6656b6d84e-openshift-service-ca\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.550541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.550134    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-maas-api-tls\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.550541 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.550175    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-metrics-certs\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.651071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.650996    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbx97\" (UniqueName: \"kubernetes.io/projected/be3ec110-72a1-428f-a093-9b6656b6d84e-kube-api-access-wbx97\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.651071 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.651052    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be3ec110-72a1-428f-a093-9b6656b6d84e-openshift-service-ca\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.651381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.651136    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-maas-api-tls\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.651381 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.651165    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-metrics-certs\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.651790 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.651759    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be3ec110-72a1-428f-a093-9b6656b6d84e-openshift-service-ca\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.653690 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.653669    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-metrics-certs\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.653848 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.653768    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be3ec110-72a1-428f-a093-9b6656b6d84e-maas-api-tls\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.708650 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.708607    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbx97\" (UniqueName: \"kubernetes.io/projected/be3ec110-72a1-428f-a093-9b6656b6d84e-kube-api-access-wbx97\") pod \"maas-api-54b8648dd4-4jn4q\" (UID: \"be3ec110-72a1-428f-a093-9b6656b6d84e\") " pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:50.749405 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:50.749362    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:51.129708 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.129494    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"]
Nov 02 22:38:51.132740 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:38:51.132706    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3ec110_72a1_428f_a093_9b6656b6d84e.slice/crio-8a29b09af9b60af3b9971cba3a4a17bce239aad4f64c57d58a482ac1624a35b4 WatchSource:0}: Error finding container 8a29b09af9b60af3b9971cba3a4a17bce239aad4f64c57d58a482ac1624a35b4: Status 404 returned error can't find the container with id 8a29b09af9b60af3b9971cba3a4a17bce239aad4f64c57d58a482ac1624a35b4
Nov 02 22:38:51.160781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.160740    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerStarted","Data":"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"}
Nov 02 22:38:51.161023 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.160805    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerStarted","Data":"3f618c4f78de8f1c726d1300721fed34987b4f3910738bf6eef3730d59b4e8fe"}
Nov 02 22:38:51.162112 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.162082    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q" event={"ID":"be3ec110-72a1-428f-a093-9b6656b6d84e","Type":"ContainerStarted","Data":"8a29b09af9b60af3b9971cba3a4a17bce239aad4f64c57d58a482ac1624a35b4"}
Nov 02 22:38:51.286282 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.286225    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podStartSLOduration=2.286209573 podStartE2EDuration="2.286209573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:38:51.285514189 +0000 UTC m=+3829.802009964" watchObservedRunningTime="2026-11-02 22:38:51.286209573 +0000 UTC m=+3829.802705347"
Nov 02 22:38:51.861233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.861197    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:51.866196 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:51.866170    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:52.169228 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:52.169195    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q" event={"ID":"be3ec110-72a1-428f-a093-9b6656b6d84e","Type":"ContainerStarted","Data":"5ba7e7888141cb560611c69e7bd518291c70c519d47b6a78ff502c8920e2fce0"}
Nov 02 22:38:52.169444 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:52.169313    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:52.169549 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:52.169529    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:52.170345 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:52.170327    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:38:52.406500 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:52.406439    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q" podStartSLOduration=1.591310496 podStartE2EDuration="2.406421367s" totalImagesPullingTime="815.110871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:38:52.288835216 +0000 UTC m=+3830.805330991" watchObservedRunningTime="2026-11-02 22:38:52.406421367 +0000 UTC m=+3830.922917135"
Nov 02 22:38:56.309884 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:38:56.309841    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:38:58.184493 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:58.184457    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54b8648dd4-4jn4q"
Nov 02 22:38:58.384702 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:58.384662    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 22:38:58.384969 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:58.384929    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" podUID="0f737e36-de9c-4020-94a8-4f7070c39163" containerName="maas-api" containerID="cri-o://86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1" gracePeriod=30
Nov 02 22:38:59.644566 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.644539    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 22:38:59.752336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752227    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\") pod \"0f737e36-de9c-4020-94a8-4f7070c39163\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") "
Nov 02 22:38:59.752336 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752270    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\" (UniqueName: \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\") pod \"0f737e36-de9c-4020-94a8-4f7070c39163\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") "
Nov 02 22:38:59.752588 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752408    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\") pod \"0f737e36-de9c-4020-94a8-4f7070c39163\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") "
Nov 02 22:38:59.752588 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752554    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\") pod \"0f737e36-de9c-4020-94a8-4f7070c39163\" (UID: \"0f737e36-de9c-4020-94a8-4f7070c39163\") "
Nov 02 22:38:59.752703 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752621    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca" pod "0f737e36-de9c-4020-94a8-4f7070c39163" (UID: "0f737e36-de9c-4020-94a8-4f7070c39163"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:38:59.753003 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.752949    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f737e36-de9c-4020-94a8-4f7070c39163-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:38:59.754557 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.754528    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls" pod "0f737e36-de9c-4020-94a8-4f7070c39163" (UID: "0f737e36-de9c-4020-94a8-4f7070c39163"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:38:59.754665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.754616    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs" pod "0f737e36-de9c-4020-94a8-4f7070c39163" (UID: "0f737e36-de9c-4020-94a8-4f7070c39163"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:38:59.754665 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.754642    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m" pod "0f737e36-de9c-4020-94a8-4f7070c39163" (UID: "0f737e36-de9c-4020-94a8-4f7070c39163"). InnerVolumeSpecName "kube-api-access-87c4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:38:59.854333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.854285    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:38:59.854333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.854326    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f737e36-de9c-4020-94a8-4f7070c39163-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:38:59.854333 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:38:59.854338    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87c4m\" (UniqueName: \"kubernetes.io/projected/0f737e36-de9c-4020-94a8-4f7070c39163-kube-api-access-87c4m\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:00.215035 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.214999    5580 generic.go:356] "Generic (PLEG): container finished" podID="0f737e36-de9c-4020-94a8-4f7070c39163" containerID="86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1" exitCode=0
Nov 02 22:39:00.215267 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.215083    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"
Nov 02 22:39:00.215267 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.215136    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" event={"ID":"0f737e36-de9c-4020-94a8-4f7070c39163","Type":"ContainerDied","Data":"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"}
Nov 02 22:39:00.215267 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.215197    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p" event={"ID":"0f737e36-de9c-4020-94a8-4f7070c39163","Type":"ContainerDied","Data":"c586310a44f28138ea041a14cac6a23e0c3af8993a92f3c786c1e778fe8bb0e7"}
Nov 02 22:39:00.215267 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.215223    5580 scope.go:122] "RemoveContainer" containerID="86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"
Nov 02 22:39:00.225717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.225697    5580 scope.go:122] "RemoveContainer" containerID="86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"
Nov 02 22:39:00.226088 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:39:00.226067    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1\": container with ID starting with 86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1 not found: ID does not exist" containerID="86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"
Nov 02 22:39:00.226172 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.226103    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1"} err="failed to get container status \"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1\": rpc error: code = NotFound desc = could not find container \"86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1\": container with ID starting with 86b413a2032da338dfe542a5083fe07574e3036ba9cb716022eb8d51ae49d6f1 not found: ID does not exist"
Nov 02 22:39:00.389094 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.389061    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 22:39:00.399900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:00.399871    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-57cf7b58d8-ppl5p"]
Nov 02 22:39:01.593274 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:01.593243    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f737e36-de9c-4020-94a8-4f7070c39163" path="/var/lib/kubelet/pods/0f737e36-de9c-4020-94a8-4f7070c39163/volumes"
Nov 02 22:39:06.158329 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:06.158284    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:39:16.299060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.299025    5580 generic.go:356] "Generic (PLEG): container finished" podID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerID="497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029" exitCode=0
Nov 02 22:39:16.299517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.299100    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerDied","Data":"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"}
Nov 02 22:39:16.299517 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.299497    5580 scope.go:122] "RemoveContainer" containerID="497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"
Nov 02 22:39:16.300788 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.300759    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:39:16.300866 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.300831    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:39:16.300866 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:16.300863    5580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 02 22:39:17.307765 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:17.307725    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerStarted","Data":"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"}
Nov 02 22:39:17.861398 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:17.861360    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:17.862645 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:17.862621    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 02 22:39:17.862709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:17.862689    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 02 22:39:18.861574 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:18.861532    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 02 22:39:18.862019 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:18.861596    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 02 22:39:19.861550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:19.861510    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 02 22:39:19.861782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:19.861581    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 02 22:39:20.861338 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:20.861295    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 02 22:39:20.861630 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:20.861354    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 02 22:39:21.861825 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:21.861780    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 02 22:39:21.862291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:21.861858    5580 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 02 22:39:22.170086 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:22.170049    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:22.866044 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:22.866013    5580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:22.867041 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:22.867017    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:26.312482 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:39:26.312437    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:39:50.715104 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:50.715060    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:39:50.715632 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:50.715365    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" containerID="cri-o://a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b" gracePeriod=30
Nov 02 22:39:51.046030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:51.045902    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:39:52.869283 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:52.869238    5580 patch_prober.go:29] interesting pod/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:39:52.869728 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:52.869309    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:39:55.968190 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.968161    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:55.992383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992295    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992342    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992383 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992365    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992640 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992497    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992640 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992588    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992640 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992629    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992648    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:39:55.992810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992662    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992680    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:39:55.992810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992753    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.992810 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992794    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\") pod \"e8bb4386-4c74-47c8-a894-358e64b747d0\" (UID: \"e8bb4386-4c74-47c8-a894-358e64b747d0\") "
Nov 02 22:39:55.993127 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992883    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:39:55.993127 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.992928    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:39:55.993247 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993206    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:39:55.993376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993353    5580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:55.993447 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993376    5580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-data\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:55.993447 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993394    5580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8bb4386-4c74-47c8-a894-358e64b747d0-istiod-ca-cert\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:55.993447 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993407    5580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-credential-socket\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:55.993447 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.993420    5580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-workload-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:55.994799 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.994771    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:39:55.994935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.994914    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:39:55.995013 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.994949    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "kube-api-access-96gh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:39:55.995074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:55.995053    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy" pod "e8bb4386-4c74-47c8-a894-358e64b747d0" (UID: "e8bb4386-4c74-47c8-a894-358e64b747d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:39:56.094213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.094165    5580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-podinfo\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:56.094213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.094206    5580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-envoy\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:56.094213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.094216    5580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-istio-token\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:56.094213 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.094225    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96gh5\" (UniqueName: \"kubernetes.io/projected/e8bb4386-4c74-47c8-a894-358e64b747d0-kube-api-access-96gh5\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:39:56.311407 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:39:56.311312    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:39:56.517717 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.517683    5580 generic.go:356] "Generic (PLEG): container finished" podID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerID="a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b" exitCode=0
Nov 02 22:39:56.517937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.517733    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerDied","Data":"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"}
Nov 02 22:39:56.517937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.517771    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr" event={"ID":"e8bb4386-4c74-47c8-a894-358e64b747d0","Type":"ContainerDied","Data":"3f618c4f78de8f1c726d1300721fed34987b4f3910738bf6eef3730d59b4e8fe"}
Nov 02 22:39:56.517937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.517792    5580 scope.go:122] "RemoveContainer" containerID="a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"
Nov 02 22:39:56.517937 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.517733    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"
Nov 02 22:39:56.527501 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.527480    5580 scope.go:122] "RemoveContainer" containerID="497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"
Nov 02 22:39:56.535995 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.535968    5580 scope.go:122] "RemoveContainer" containerID="a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"
Nov 02 22:39:56.536326 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:39:56.536306    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b\": container with ID starting with a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b not found: ID does not exist" containerID="a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"
Nov 02 22:39:56.536376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.536339    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b"} err="failed to get container status \"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b\": rpc error: code = NotFound desc = could not find container \"a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b\": container with ID starting with a33bb0cce5fd3c3bf347ce93feaa04da23df20197ded0f1577bbd8d9294f506b not found: ID does not exist"
Nov 02 22:39:56.536376 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.536357    5580 scope.go:122] "RemoveContainer" containerID="497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"
Nov 02 22:39:56.536662 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:39:56.536644    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029\": container with ID starting with 497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029 not found: ID does not exist" containerID="497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"
Nov 02 22:39:56.536700 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.536669    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029"} err="failed to get container status \"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029\": rpc error: code = NotFound desc = could not find container \"497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029\": container with ID starting with 497b93ea73c39e0dc0e7ed7e39268ab53d3035da63a24bd5c7e3f4c01f8b4029 not found: ID does not exist"
Nov 02 22:39:56.685724 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.685690    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:39:56.735415 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:56.735375    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ce65d173-openshift-default-5b947847b5-xnzlr"]
Nov 02 22:39:57.597994 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:39:57.595546    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" path="/var/lib/kubelet/pods/e8bb4386-4c74-47c8-a894-358e64b747d0/volumes"
Nov 02 22:40:00.113140 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113101    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:40:00.113550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113520    5580 cpu_manager.go:414] "containerMap: removing container" podUID="0f737e36-de9c-4020-94a8-4f7070c39163" containerName="maas-api"
Nov 02 22:40:00.113550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113531    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f737e36-de9c-4020-94a8-4f7070c39163" containerName="maas-api"
Nov 02 22:40:00.113550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113544    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.113550 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113550    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.113685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113653    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f737e36-de9c-4020-94a8-4f7070c39163" containerName="maas-api"
Nov 02 22:40:00.113685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113664    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.113685 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.113675    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.118626 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.118605    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.134282 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.134247    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.134449 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.134387    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxq2\" (UniqueName: \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.174564 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.174530    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:40:00.221901 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.221865    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-0f09fe\""
Nov 02 22:40:00.222126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.222004    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-0f09fe-dockercfg-fp7wk\""
Nov 02 22:40:00.235074 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.235036    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.235285 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.235131    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxq2\" (UniqueName: \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.235731 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.235709    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.273126 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.273092    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:40:00.273546 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.273533    5580 cpu_manager.go:414] "containerMap: removing container" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.273597 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.273547    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8bb4386-4c74-47c8-a894-358e64b747d0" containerName="istio-proxy"
Nov 02 22:40:00.279928 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.279902    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.291815 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.291790    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxq2\" (UniqueName: \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\") pod \"payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.307653 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.307567    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:40:00.336856 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.336811    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t25j\" (UniqueName: \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.337129 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.337013    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.432206 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.432173    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:00.438476 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.438450    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t25j\" (UniqueName: \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.438562 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.438547    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.439233 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.439212    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.495225 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.495191    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t25j\" (UniqueName: \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\") pod \"payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.594610 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.594566    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:00.803263 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.803155    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:40:00.806275 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:40:00.806247    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1688374d_7ebc_46a7_995e_99e0f90e90a2.slice/crio-405fd4c3f2cda14fdb3b97854a67cec7771feeec19aec8cc8ff99d4830acecf1 WatchSource:0}: Error finding container 405fd4c3f2cda14fdb3b97854a67cec7771feeec19aec8cc8ff99d4830acecf1: Status 404 returned error can't find the container with id 405fd4c3f2cda14fdb3b97854a67cec7771feeec19aec8cc8ff99d4830acecf1
Nov 02 22:40:00.971935 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:00.971906    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:40:00.974313 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:40:00.974281    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9674679_4b1e_4aa8_bf14_a5baa1754aaa.slice/crio-4180066fdaad15b21618be3dd92a214c73c386563244a9b716f76780590485a4 WatchSource:0}: Error finding container 4180066fdaad15b21618be3dd92a214c73c386563244a9b716f76780590485a4: Status 404 returned error can't find the container with id 4180066fdaad15b21618be3dd92a214c73c386563244a9b716f76780590485a4
Nov 02 22:40:01.544220 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.544176    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" event={"ID":"d9674679-4b1e-4aa8-bf14-a5baa1754aaa","Type":"ContainerStarted","Data":"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"}
Nov 02 22:40:01.544701 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.544263    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:01.544701 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.544294    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" event={"ID":"d9674679-4b1e-4aa8-bf14-a5baa1754aaa","Type":"ContainerStarted","Data":"4180066fdaad15b21618be3dd92a214c73c386563244a9b716f76780590485a4"}
Nov 02 22:40:01.545939 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.545906    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" event={"ID":"1688374d-7ebc-46a7-995e-99e0f90e90a2","Type":"ContainerStarted","Data":"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"}
Nov 02 22:40:01.546116 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.545967    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:01.546116 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.545986    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" event={"ID":"1688374d-7ebc-46a7-995e-99e0f90e90a2","Type":"ContainerStarted","Data":"405fd4c3f2cda14fdb3b97854a67cec7771feeec19aec8cc8ff99d4830acecf1"}
Nov 02 22:40:01.657388 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.657334    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" podStartSLOduration=1.65731601 podStartE2EDuration="1.65731601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:40:01.656333807 +0000 UTC m=+3900.172829578" watchObservedRunningTime="2026-11-02 22:40:01.65731601 +0000 UTC m=+3900.173811785"
Nov 02 22:40:01.771408 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:01.771345    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" podStartSLOduration=1.771324592 podStartE2EDuration="1.771324592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:40:01.768716165 +0000 UTC m=+3900.285211965" watchObservedRunningTime="2026-11-02 22:40:01.771324592 +0000 UTC m=+3900.287820367"
Nov 02 22:40:02.360131 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:02.360104    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:40:02.370910 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:02.370886    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:40:02.388150 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:02.388118    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:40:02.398485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:02.398453    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:40:03.048848 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.048813    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:40:03.052601 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.052583    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.157423 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.157390    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-0f09fe-serving-cert\""
Nov 02 22:40:03.159137 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.159114    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:40:03.168176 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.168145    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.168293 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.168210    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.168370 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.168350    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.168434 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.168409    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4lb\" (UniqueName: \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.269941 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.269894    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.270177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.270016    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4lb\" (UniqueName: \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.270177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.270070    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.270177 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.270137    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.270316 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:40:03.270298    5580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-serving-cert: secret "maas-api-e2e-authcfg-http-0f09fe-serving-cert" not found
Nov 02 22:40:03.270402 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:40:03.270390    5580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls podName:bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243 nodeName:}" failed. No retries permitted until 2026-11-02 22:40:03.770352 +0000 UTC m=+3902.286847753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls") pod "maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" (UID: "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243") : secret "maas-api-e2e-authcfg-http-0f09fe-serving-cert" not found
Nov 02 22:40:03.270756 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.270736    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.272577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.272558    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.327842 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.327763    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4lb\" (UniqueName: \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.776445 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.776407    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.778868 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.778839    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:03.965562 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:03.965527    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:04.364781 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:04.364752    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:40:04.367668 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:40:04.367634    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbccee3_d66e_4b5b_95cb_1ce1ae0c5243.slice/crio-5746938fe00660b58db992487ea5ac230b8a1a044442f853c601c9e516affda8 WatchSource:0}: Error finding container 5746938fe00660b58db992487ea5ac230b8a1a044442f853c601c9e516affda8: Status 404 returned error can't find the container with id 5746938fe00660b58db992487ea5ac230b8a1a044442f853c601c9e516affda8
Nov 02 22:40:04.369188 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:04.369166    5580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:40:04.562796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:04.562755    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" event={"ID":"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243","Type":"ContainerStarted","Data":"5746938fe00660b58db992487ea5ac230b8a1a044442f853c601c9e516affda8"}
Nov 02 22:40:05.570325 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:05.570233    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" event={"ID":"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243","Type":"ContainerStarted","Data":"ff87db6ba5e9c8bdb2c0744f3345bcf3e6191a056819cd5ad3588739d884a466"}
Nov 02 22:40:05.570705 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:05.570359    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:05.687302 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:05.687243    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" podStartSLOduration=1.792123348 podStartE2EDuration="2.687228295s" totalImagesPullingTime="895.104947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:40:05.685596928 +0000 UTC m=+3904.202092703" watchObservedRunningTime="2026-11-02 22:40:05.687228295 +0000 UTC m=+3904.203724111"
Nov 02 22:40:11.062677 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:11.062641    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:40:11.581465 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:11.581435    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:40:12.555392 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:12.555363    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:40:12.555877 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:40:12.555441    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:40:26.310930 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:40:26.310885    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:40:56.311205 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:40:56.311165    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:41:26.311286 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:41:26.311244    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:41:55.725437 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:41:55.725400    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:41:55.829030 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:41:55.828997    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:41:56.311499 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:41:56.311456    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:42:00.725860 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:00.725807    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" podUID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" containerName="payload-processing" containerID="cri-o://605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15" gracePeriod=25
Nov 02 22:42:00.830252 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:00.830186    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" podUID="1688374d-7ebc-46a7-995e-99e0f90e90a2" containerName="payload-pre-processing" containerID="cri-o://d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c" gracePeriod=25
Nov 02 22:42:01.001226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.001196    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:42:01.053683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.053642    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\" (UniqueName: \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\") pod \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") "
Nov 02 22:42:01.053900 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.053725    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\") pod \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\" (UID: \"d9674679-4b1e-4aa8-bf14-a5baa1754aaa\") "
Nov 02 22:42:01.054165 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.054142    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume" pod "d9674679-4b1e-4aa8-bf14-a5baa1754aaa" (UID: "d9674679-4b1e-4aa8-bf14-a5baa1754aaa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:42:01.056384 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.056360    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j" pod "d9674679-4b1e-4aa8-bf14-a5baa1754aaa" (UID: "d9674679-4b1e-4aa8-bf14-a5baa1754aaa"). InnerVolumeSpecName "kube-api-access-2t25j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:42:01.082594 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.082565    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:42:01.154713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.154658    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\" (UniqueName: \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\") pod \"1688374d-7ebc-46a7-995e-99e0f90e90a2\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") "
Nov 02 22:42:01.154713 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.154717    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\") pod \"1688374d-7ebc-46a7-995e-99e0f90e90a2\" (UID: \"1688374d-7ebc-46a7-995e-99e0f90e90a2\") "
Nov 02 22:42:01.155042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.155021    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t25j\" (UniqueName: \"kubernetes.io/projected/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-kube-api-access-2t25j\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:01.155042 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.155042    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9674679-4b1e-4aa8-bf14-a5baa1754aaa-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:01.155223 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.155193    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume" pod "1688374d-7ebc-46a7-995e-99e0f90e90a2" (UID: "1688374d-7ebc-46a7-995e-99e0f90e90a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:42:01.156987 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.156950    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2" pod "1688374d-7ebc-46a7-995e-99e0f90e90a2" (UID: "1688374d-7ebc-46a7-995e-99e0f90e90a2"). InnerVolumeSpecName "kube-api-access-nkxq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:42:01.196016 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.195982    5580 generic.go:356] "Generic (PLEG): container finished" podID="1688374d-7ebc-46a7-995e-99e0f90e90a2" containerID="d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c" exitCode=0
Nov 02 22:42:01.196226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.196081    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"
Nov 02 22:42:01.196226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.196109    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" event={"ID":"1688374d-7ebc-46a7-995e-99e0f90e90a2","Type":"ContainerDied","Data":"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"}
Nov 02 22:42:01.196226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.196168    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz" event={"ID":"1688374d-7ebc-46a7-995e-99e0f90e90a2","Type":"ContainerDied","Data":"405fd4c3f2cda14fdb3b97854a67cec7771feeec19aec8cc8ff99d4830acecf1"}
Nov 02 22:42:01.196226 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.196199    5580 scope.go:122] "RemoveContainer" containerID="d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"
Nov 02 22:42:01.197448 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.197426    5580 generic.go:356] "Generic (PLEG): container finished" podID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" containerID="605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15" exitCode=0
Nov 02 22:42:01.197536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.197475    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"
Nov 02 22:42:01.197536 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.197487    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" event={"ID":"d9674679-4b1e-4aa8-bf14-a5baa1754aaa","Type":"ContainerDied","Data":"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"}
Nov 02 22:42:01.197658 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.197539    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm" event={"ID":"d9674679-4b1e-4aa8-bf14-a5baa1754aaa","Type":"ContainerDied","Data":"4180066fdaad15b21618be3dd92a214c73c386563244a9b716f76780590485a4"}
Nov 02 22:42:01.206371 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.206352    5580 scope.go:122] "RemoveContainer" containerID="d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"
Nov 02 22:42:01.206708 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:42:01.206685    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c\": container with ID starting with d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c not found: ID does not exist" containerID="d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"
Nov 02 22:42:01.206797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.206719    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c"} err="failed to get container status \"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c\": rpc error: code = NotFound desc = could not find container \"d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c\": container with ID starting with d8aa7fb6375fe511294e3a2a96f1eacecec81fb1500414a5b0d2cb7a1f85ea8c not found: ID does not exist"
Nov 02 22:42:01.206797 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.206741    5580 scope.go:122] "RemoveContainer" containerID="605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"
Nov 02 22:42:01.218291 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.217485    5580 scope.go:122] "RemoveContainer" containerID="605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"
Nov 02 22:42:01.218662 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:42:01.218636    5580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15\": container with ID starting with 605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15 not found: ID does not exist" containerID="605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"
Nov 02 22:42:01.218746 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.218673    5580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15"} err="failed to get container status \"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15\": rpc error: code = NotFound desc = could not find container \"605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15\": container with ID starting with 605906bd9ac5b9a442f80734803b1cd770abbe8c682c4c28d0205d0aa9cb9f15 not found: ID does not exist"
Nov 02 22:42:01.256179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.256084    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkxq2\" (UniqueName: \"kubernetes.io/projected/1688374d-7ebc-46a7-995e-99e0f90e90a2-kube-api-access-nkxq2\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:01.256179 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.256119    5580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1688374d-7ebc-46a7-995e-99e0f90e90a2-plugins-config-volume\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:01.336454 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.336411    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:42:01.341209 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.341172    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-0f09fe-7b6bd7ccdc-7wrlz"]
Nov 02 22:42:01.457440 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.457407    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:42:01.459397 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.459362    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-0f09fe-7d7d48695f-9fncm"]
Nov 02 22:42:01.592938 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.592858    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1688374d-7ebc-46a7-995e-99e0f90e90a2" path="/var/lib/kubelet/pods/1688374d-7ebc-46a7-995e-99e0f90e90a2/volumes"
Nov 02 22:42:01.593463 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:01.593443    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" path="/var/lib/kubelet/pods/d9674679-4b1e-4aa8-bf14-a5baa1754aaa/volumes"
Nov 02 22:42:02.977485 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:02.977449    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:42:02.977867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:02.977672    5580 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" podUID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" containerName="maas-api" containerID="cri-o://ff87db6ba5e9c8bdb2c0744f3345bcf3e6191a056819cd5ad3588739d884a466" gracePeriod=30
Nov 02 22:42:03.211761 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.211725    5580 generic.go:356] "Generic (PLEG): container finished" podID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" containerID="ff87db6ba5e9c8bdb2c0744f3345bcf3e6191a056819cd5ad3588739d884a466" exitCode=0
Nov 02 22:42:03.211950 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.211799    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" event={"ID":"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243","Type":"ContainerDied","Data":"ff87db6ba5e9c8bdb2c0744f3345bcf3e6191a056819cd5ad3588739d884a466"}
Nov 02 22:42:03.251105 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.251077    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:42:03.275678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.275641    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\") pod \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") "
Nov 02 22:42:03.275865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.275762    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") pod \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") "
Nov 02 22:42:03.275865 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.275800    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\" (UniqueName: \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\") pod \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") "
Nov 02 22:42:03.275991 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.275904    5580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\") pod \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\" (UID: \"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243\") "
Nov 02 22:42:03.276323 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.276286    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca" pod "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" (UID: "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:42:03.278230 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.278197    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb" pod "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" (UID: "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243"). InnerVolumeSpecName "kube-api-access-vx4lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:42:03.278448 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.278415    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls" pod "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" (UID: "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:42:03.278448 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.278436    5580 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs" pod "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" (UID: "bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:42:03.377507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.377473    5580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-openshift-service-ca\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:03.377507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.377504    5580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-metrics-certs\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:03.377507 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.377514    5580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-maas-api-tls\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:03.377795 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:03.377522    5580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx4lb\" (UniqueName: \"kubernetes.io/projected/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243-kube-api-access-vx4lb\") on node \"ip-10-0-136-74.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 22:42:04.217851 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:04.217809    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg" event={"ID":"bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243","Type":"ContainerDied","Data":"5746938fe00660b58db992487ea5ac230b8a1a044442f853c601c9e516affda8"}
Nov 02 22:42:04.218344 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:04.217871    5580 scope.go:122] "RemoveContainer" containerID="ff87db6ba5e9c8bdb2c0744f3345bcf3e6191a056819cd5ad3588739d884a466"
Nov 02 22:42:04.218344 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:04.217823    5580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"
Nov 02 22:42:04.381420 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:04.381385    5580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:42:04.430796 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:04.430765    5580 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-0f09fe-7d5f7b8b8-68zvg"]
Nov 02 22:42:05.592402 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:05.592366    5580 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" path="/var/lib/kubelet/pods/bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243/volumes"
Nov 02 22:42:09.211543 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:09.211501    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:42:18.306981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:18.306058    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-vdlkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:42:18.306981 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:18.306140    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-vdlkp" podUID="97743216-5aa5-42fb-a341-718c72c14b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:42:18.489149 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:18.489101    5580 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-65c448b547-trltt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:42:18.489425 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:18.489395    5580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-65c448b547-trltt" podUID="5fdd3a32-f031-438c-a5a5-cae4b101b2e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:42:21.930061 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:21.930023    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:42:26.311481 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:42:26.311440    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:42:34.131625 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:34.131586    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:42:54.931663 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:42:54.931626    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:42:56.311027 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:42:56.310987    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:43:26.310888 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:43:26.310850    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:43:26.961341 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:26.961301    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hkqn9"]
Nov 02 22:43:37.298749 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:37.298713    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-cvqrl_494cbe8e-607d-4312-9a4f-242ad7930133/manager/0.log"
Nov 02 22:43:37.577340 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:37.577261    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-f9srl_dffd3bb3-8e6a-49da-873a-d182ce1c70c9/manager/0.log"
Nov 02 22:43:38.403319 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:38.403288    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:43:39.092259 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:39.092224    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-v62z6_86069694-d120-4b17-9a6d-3cb3ac51595b/manager/0.log"
Nov 02 22:43:40.150873 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:40.150841    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54b8648dd4-4jn4q_be3ec110-72a1-428f-a093-9b6656b6d84e/maas-api/0.log"
Nov 02 22:43:43.685330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:43.685303    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-pwks5_a21ad6c4-0ddc-43c1-affa-d7c953dadc8b/kuadrant-console-plugin/0.log"
Nov 02 22:43:44.674146 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:44.674118    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hkqn9_e8897fab-c0b8-4b3c-85e2-4b05af9959cc/limitador/0.log"
Nov 02 22:43:46.395193 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:46.395161    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d64f9d85b-qz28l_805968c9-e4bc-4a55-8c25-08ddfd3c2234/kube-auth-proxy/0.log"
Nov 02 22:43:46.886762 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:46.886725    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-dnxdq_95921fe2-1e3d-4fdd-a833-d4fbef78da76/istio-proxy/0.log"
Nov 02 22:43:47.955330 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:47.955303    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-trltt_5fdd3a32-f031-438c-a5a5-cae4b101b2e1/istio-proxy/0.log"
Nov 02 22:43:48.203867 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:48.203837    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-v7sg6_0dfaaa24-2c65-4d37-96e2-82b16948e324/istio-proxy/0.log"
Nov 02 22:43:48.461709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:48.461680    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-vdlkp_97743216-5aa5-42fb-a341-718c72c14b4f/istio-proxy/0.log"
Nov 02 22:43:49.195173 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:49.195142    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd_ca908a18-62ad-40f8-9929-190590cb62ea/istio-proxy/0.log"
Nov 02 22:43:50.877392 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:50.877363    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-g2h6s_e267af0e-0894-4514-815d-3bed98989354/main/0.log"
Nov 02 22:43:51.350772 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:51.350744    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-gvpnj_108043cb-e370-4bb2-961e-191c3414ab62/main/0.log"
Nov 02 22:43:51.817584 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:51.817555    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4txnq_bb30c5fd-fa38-4323-ad59-8888963b7224/main/0.log"
Nov 02 22:43:52.292240 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:43:52.292206    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc_9b51a5e6-f2f4-4349-bf3e-47774cc29115/main/0.log"
Nov 02 22:43:56.310639 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:43:56.310601    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:44:02.225784 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:02.225750    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-cvqrl_494cbe8e-607d-4312-9a4f-242ad7930133/manager/0.log"
Nov 02 22:44:02.497788 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:02.497698    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-f9srl_dffd3bb3-8e6a-49da-873a-d182ce1c70c9/manager/0.log"
Nov 02 22:44:03.240733 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:03.240695    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:44:03.778089 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:03.778053    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-v62z6_86069694-d120-4b17-9a6d-3cb3ac51595b/manager/0.log"
Nov 02 22:44:04.596897 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:04.596858    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54b8648dd4-4jn4q_be3ec110-72a1-428f-a093-9b6656b6d84e/maas-api/0.log"
Nov 02 22:44:07.716904 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:07.716877    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-pwks5_a21ad6c4-0ddc-43c1-affa-d7c953dadc8b/kuadrant-console-plugin/0.log"
Nov 02 22:44:08.384439 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:08.384408    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hkqn9_e8897fab-c0b8-4b3c-85e2-4b05af9959cc/limitador/0.log"
Nov 02 22:44:09.961678 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:09.961646    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d64f9d85b-qz28l_805968c9-e4bc-4a55-8c25-08ddfd3c2234/kube-auth-proxy/0.log"
Nov 02 22:44:10.455363 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:10.455314    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-dnxdq_95921fe2-1e3d-4fdd-a833-d4fbef78da76/istio-proxy/0.log"
Nov 02 22:44:11.544577 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:11.544543    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-trltt_5fdd3a32-f031-438c-a5a5-cae4b101b2e1/istio-proxy/0.log"
Nov 02 22:44:11.821391 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:11.821273    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-v7sg6_0dfaaa24-2c65-4d37-96e2-82b16948e324/istio-proxy/0.log"
Nov 02 22:44:12.037944 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:12.037909    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-65c448b547-vdlkp_97743216-5aa5-42fb-a341-718c72c14b4f/istio-proxy/0.log"
Nov 02 22:44:12.804263 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:12.804228    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-69fd6b667f-jw9nd_ca908a18-62ad-40f8-9929-190590cb62ea/istio-proxy/0.log"
Nov 02 22:44:14.423902 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:14.423868    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-g2h6s_e267af0e-0894-4514-815d-3bed98989354/main/0.log"
Nov 02 22:44:14.952699 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:14.952659    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-gvpnj_108043cb-e370-4bb2-961e-191c3414ab62/main/0.log"
Nov 02 22:44:15.437143 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:15.437109    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4txnq_bb30c5fd-fa38-4323-ad59-8888963b7224/main/0.log"
Nov 02 22:44:15.943404 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:15.943361    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-mbxxc_9b51a5e6-f2f4-4349-bf3e-47774cc29115/main/0.log"
Nov 02 22:44:26.311111 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:44:26.311070    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:44:46.174520 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:46.174485    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:44:46.695204 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:44:46.695161    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:44:56.310756 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:44:56.310716    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:45:02.393366 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:02.393336    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:45:02.403709 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:02.403681    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:45:02.421598 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:02.421575    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:45:02.431828 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:02.431801    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pcn6c_3dc7319d-94df-4f5d-88bb-9b85e9ee0eb7/ovn-acl-logging/0.log"
Nov 02 22:45:23.950755 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:23.950711    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:45:24.516141 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:24.516102    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5cc9f7769b-j4bwf_3065b811-5eab-4780-bbb7-8302e87bb1b8/manager/0.log"
Nov 02 22:45:26.313290 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:45:26.313249    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:45:34.896622 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:34.896586    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ljkpr_2e177cbc-5f97-45c8-9714-73510ec5597b/global-pull-secret-syncer/0.log"
Nov 02 22:45:38.532762 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:38.532723    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wc9ll_ab35d4ec-3432-43b7-adca-34097c507ae1/konnectivity-agent/0.log"
Nov 02 22:45:40.228911 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:40.228880    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-74.us-west-1.compute.internal_8ed0968c21495795a0292b52f44b9b55/haproxy/0.log"
Nov 02 22:45:44.862046 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.861942    5580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"]
Nov 02 22:45:44.862472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862432    5580 cpu_manager.go:414] "containerMap: removing container" podUID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" containerName="payload-processing"
Nov 02 22:45:44.862472 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862447    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" containerName="payload-processing"
Nov 02 22:45:44.862555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862479    5580 cpu_manager.go:414] "containerMap: removing container" podUID="1688374d-7ebc-46a7-995e-99e0f90e90a2" containerName="payload-pre-processing"
Nov 02 22:45:44.862555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862485    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1688374d-7ebc-46a7-995e-99e0f90e90a2" containerName="payload-pre-processing"
Nov 02 22:45:44.862555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862510    5580 cpu_manager.go:414] "containerMap: removing container" podUID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" containerName="maas-api"
Nov 02 22:45:44.862555 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862515    5580 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" containerName="maas-api"
Nov 02 22:45:44.862683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862573    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="1688374d-7ebc-46a7-995e-99e0f90e90a2" containerName="payload-pre-processing"
Nov 02 22:45:44.862683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862587    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcbccee3-d66e-4b5b-95cb-1ce1ae0c5243" containerName="maas-api"
Nov 02 22:45:44.862683 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.862593    5580 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9674679-4b1e-4aa8-bf14-a5baa1754aaa" containerName="payload-processing"
Nov 02 22:45:44.865854 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.865830    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:44.967026 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.966992    5580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lxt6h\"/\"default-dockercfg-q6zgg\""
Nov 02 22:45:44.967231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.967068    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lxt6h\"/\"kube-root-ca.crt\""
Nov 02 22:45:44.967231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.967172    5580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lxt6h\"/\"openshift-service-ca.crt\""
Nov 02 22:45:44.968674 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.968653    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"]
Nov 02 22:45:44.981223 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.981190    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-proc\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:44.981413 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.981265    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-lib-modules\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:44.981413 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.981329    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-sys\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:44.981519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.981463    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-podres\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:44.981519 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:44.981506    5580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzr4\" (UniqueName: \"kubernetes.io/projected/12b77b8c-ada0-4345-9522-2d6d81ffedad-kube-api-access-zfzr4\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082484 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082430    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-sys\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082546    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-podres\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082559    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-sys\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082596    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzr4\" (UniqueName: \"kubernetes.io/projected/12b77b8c-ada0-4345-9522-2d6d81ffedad-kube-api-access-zfzr4\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082630    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-podres\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082661    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-proc\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.082720 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082692    5580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-lib-modules\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.083121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082770    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-proc\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.083121 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.082814    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b77b8c-ada0-4345-9522-2d6d81ffedad-lib-modules\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.139157 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.139083    5580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzr4\" (UniqueName: \"kubernetes.io/projected/12b77b8c-ada0-4345-9522-2d6d81ffedad-kube-api-access-zfzr4\") pod \"perf-node-gather-daemonset-6wdkr\" (UID: \"12b77b8c-ada0-4345-9522-2d6d81ffedad\") " pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.178776 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.178734    5580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:45.415477 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.415394    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-pwks5_a21ad6c4-0ddc-43c1-affa-d7c953dadc8b/kuadrant-console-plugin/0.log"
Nov 02 22:45:45.553289 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:45.553258    5580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"]
Nov 02 22:45:45.556153 ip-10-0-136-74 kubenswrapper[5580]: W1102 22:45:45.556127    5580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod12b77b8c_ada0_4345_9522_2d6d81ffedad.slice/crio-c08a0e1ea9a29a5f5a315d12cb06331cfa419441530967da2ff6489e2c11112b WatchSource:0}: Error finding container c08a0e1ea9a29a5f5a315d12cb06331cfa419441530967da2ff6489e2c11112b: Status 404 returned error can't find the container with id c08a0e1ea9a29a5f5a315d12cb06331cfa419441530967da2ff6489e2c11112b
Nov 02 22:45:46.404492 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:46.404455    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr" event={"ID":"12b77b8c-ada0-4345-9522-2d6d81ffedad","Type":"ContainerStarted","Data":"9d6aefd2ea1b3783fa03610f61d63975341f137db0c3e97560e3f07f08ede294"}
Nov 02 22:45:46.404921 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:46.404542    5580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr" event={"ID":"12b77b8c-ada0-4345-9522-2d6d81ffedad","Type":"ContainerStarted","Data":"c08a0e1ea9a29a5f5a315d12cb06331cfa419441530967da2ff6489e2c11112b"}
Nov 02 22:45:46.404921 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:46.404596    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:46.516886 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:46.516823    5580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr" podStartSLOduration=2.516807249 podStartE2EDuration="2.516807249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:45:46.516516348 +0000 UTC m=+4245.033012122" watchObservedRunningTime="2026-11-02 22:45:46.516807249 +0000 UTC m=+4245.033303049"
Nov 02 22:45:46.547329 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:46.547304    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hkqn9_e8897fab-c0b8-4b3c-85e2-4b05af9959cc/limitador/0.log"
Nov 02 22:45:52.419738 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:52.419697    5580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lxt6h/perf-node-gather-daemonset-6wdkr"
Nov 02 22:45:52.924947 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:52.924913    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-f5w8b_4e1a1df8-8a6a-4136-9444-9eb795619aed/kube-state-metrics/0.log"
Nov 02 22:45:53.380359 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:53.380277    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-f5w8b_4e1a1df8-8a6a-4136-9444-9eb795619aed/kube-rbac-proxy-main/0.log"
Nov 02 22:45:53.791934 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:53.791887    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-f5w8b_4e1a1df8-8a6a-4136-9444-9eb795619aed/kube-rbac-proxy-self/0.log"
Nov 02 22:45:54.625309 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:54.625277    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-tvxmt_e58fab09-a42f-4f63-96a9-cfe58f7ed15c/monitoring-plugin/0.log"
Nov 02 22:45:56.298254 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:56.298226    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gg8vl_459afbc7-52e2-42f9-98f0-b157dfc74671/node-exporter/0.log"
Nov 02 22:45:56.311476 ip-10-0-136-74 kubenswrapper[5580]: E1102 22:45:56.311441    5580 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9hsfk" containerName="extractor"
Nov 02 22:45:56.708811 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:56.708780    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gg8vl_459afbc7-52e2-42f9-98f0-b157dfc74671/kube-rbac-proxy/0.log"
Nov 02 22:45:57.122192 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:57.122112    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gg8vl_459afbc7-52e2-42f9-98f0-b157dfc74671/init-textfile/0.log"
Nov 02 22:45:58.792703 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:58.792667    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sxq8l_a3f84beb-14a1-406c-924e-3896b29f6564/kube-rbac-proxy-main/0.log"
Nov 02 22:45:59.204443 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:59.204412    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sxq8l_a3f84beb-14a1-406c-924e-3896b29f6564/kube-rbac-proxy-self/0.log"
Nov 02 22:45:59.617406 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:45:59.617311    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sxq8l_a3f84beb-14a1-406c-924e-3896b29f6564/openshift-state-metrics/0.log"
Nov 02 22:46:00.040798 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:00.040734    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/0.log"
Nov 02 22:46:00.042672 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:00.042652    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/prometheus/1.log"
Nov 02 22:46:00.352660 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:00.352576    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/config-reloader/0.log"
Nov 02 22:46:00.765737 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:00.765699    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/thanos-sidecar/0.log"
Nov 02 22:46:01.179205 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:01.179171    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/kube-rbac-proxy-web/0.log"
Nov 02 22:46:01.593060 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:01.592987    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/kube-rbac-proxy/0.log"
Nov 02 22:46:02.008163 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:02.008114    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/kube-rbac-proxy-thanos/0.log"
Nov 02 22:46:02.425335 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:02.425311    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddb36de-40b5-40e8-a49c-7ddcb7d98550/init-config-reloader/0.log"
Nov 02 22:46:05.330848 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:05.330815    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/thanos-query/0.log"
Nov 02 22:46:05.741782 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:05.741753    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/kube-rbac-proxy-web/0.log"
Nov 02 22:46:06.156231 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:06.156143    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/kube-rbac-proxy/0.log"
Nov 02 22:46:06.570741 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:06.570714    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/prom-label-proxy/0.log"
Nov 02 22:46:06.983822 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:06.983793    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/kube-rbac-proxy-rules/0.log"
Nov 02 22:46:07.414901 ip-10-0-136-74 kubenswrapper[5580]: I1102 22:46:07.414824    5580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74fc7f6c58-k7tvj_3be53397-eac6-47b3-8b63-faa5479e1f3b/kube-rbac-proxy-metrics/0.log"